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1.  INTRODUCTION  I 

i 

Weapons  Research  Establishment  installed  an  IBM  370/168  \ 

computer  in  September  1975.  As  part  of  the  preparation  that  i 

preceded  this  event  an  automatic  data  migration  scheme  was  | 

designed  and  developed.  j 

The  requirement  for  such  a scheme  was  evident  from  the  i 

experiences  of  other  large  computer  installations  with  extensive  j 

interactive  terminal  facilities  - like  those  at  W.R.E.  Most  1 

installations  found  it  difficult  to  control  the  disk  storage  j 

space  allocated  for  user  data.  As  a result  the  disks  soon  ’ 

filled/  creating  a situation  which  required  considerable  effort 
to  resolve.  This  is  because  users  are  reluctant  to  delete  data  | 

sets  no  longer  required  and  because  any  alternative  to  disk 
storage  is  inconvenient. 

An  automatic  data  migration  scheme  goes  a long  way  towards 
providing  a solution.  It  gives  control  over  the  amount  of  free 
space  available  on  the  disks  for  the  storage  of  new  data.  In  our 
scheme  space  Is  released  by  the  automatic  transfer  of  selected 
datasets  from  disk  to  magnetic  tape.  Should  a dataset  be 
required  for  further  processing  it  can  easily  be  returned  to 
disk.  This  is  not  performed  automatically/  but  can  be  initiated 
by  simple  user  action.  References  1/  2 an4(  3 describe  similar 

schemes  used  at  other  installations.  However  none  of  these  were 
particularly  suited  to  our  environment  and  requirements. 

At  present  the  migration  scheme  handles  two  of  the  dataset 
organizations  supported  by  IBM's  operating  system.  They  are  the  ' 

sequential  and  partitioned  organizations/  which  account  for  over 
90%  of  user  space  occupancy.  Note  however  that  datasets  with  the  i 

track-overflow  feature  cannot  be  handled  and  an  installation 
standard  prohibits  the  use  of  this  data  format. 

The  migration  scheme  is  complemented  by  a tape  management  \ 

system  for  handling  very  large  sequential  files.  This  offers  a j 

combined  service  comparable  with  that  provided  to  FORTRAN  users  \ 

on  our  previous  general  purpose  computer/  an  IBM  7090/  which  used  j 

magnetic  tape  for  all  data  storage. 

The  7090  data  management  system  maintained  30000  user  files  on 
5000  reels  of  tape.  Every  data  record  in  this  system  was  I 

uniquely  labelled  without  user  action  or  programming.  User  files  j: 

could  be  written  sequentially  on  the  same  tape  and  continued  onto 
any  empty  labelled  spool  automatically.  By  a process  of 
selective  transcription  and  a family  of  computer  programs  the 
operating  staff  maintained  the  tape  library  at  a manageable  size 
and  expunged  obsolete  files. 

A mathematical  model  of  the  migration  scheme  was  developed 
concurrently  with  its  actual  implementation(ref .13) . All 
parameters  available  for  manipulating  the  migration  scheme  have 
been  included  as  input  to  the  mode’.  Our  intention  is  to  use  the 
model  to  analyse  possible  effects  of  changing  an  input  parameter 
on  the  operation  of  the  real  system.  This  will  enable  us  to  make 
better  decisions  regarding  parameter  changes  without  waiting  long 
periods  to  observe  the  results. 
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ThrouRhout  this  document  the  term  migration  refers  to  the 
controlled  transfer  of  diita  from  disk  to  moRnetlc  tape  by  the 
computing  centre.  The  term  archival  refers  to  the  iiiHnediate 
transfer  of  data  to  magnetic  tape  as  the  result  of  a user 
request . 


2.  ADVANTAGES  OF  DATA  MIGRATION 

Since  the  Implementation  of  a scheme  for  automatic  data 
migration  requires  a significant  amount  of  programming  effort 
other  approaches  to  controlling  the  use  of  disk  storage  were 
considered.  For  example,  one  could  impose  a limit  on  the  amount 
of  disk  space  available  to  each  user.  This  raises  the  question 
of  how  violations  are  to  be  treated  and  the  experiences  of  others 
suggest  that  the  success  of  this  approach  Is  limited  and  an 
excessive  amount  of  manual  Intervention  Is  necessary.  Data 
migration  allows  automatic  control  of  disk  space  and  users  are 
only  required  to  maintain  a record  of  the  names  of  their 
datasets.  A simple  approach  is  essential  In  an  installation 
supporting  500  users  of  all  grades  of  computing  skill  and 
requ I rements . 


t 


r 


3.  GENERAL  DATASET  MANAGEMENT 


3.1  Conventions 

The  Installation  has  adopted  certain  dataset  management 
conventions  that  users  are  required  ' to  follow.  These  are 
described  below. 

(a)  Dataset  names  must  reflect  their  owner.  Each  user  of 
the  computer  system  has  a unique  three  character 
Identification  code  which  Is  known  as  the  userid. 
This  code  must  appear  as  the  first  qualifier  In  the 
name  of  every  dataset  created  by  the  user. 

(b)  Each  permanent  dataset  must  be  catalogued  on 
creation.  For  subsequent  access  the  user  only 
supplies  the  dataset  name.  The  catalogue  contains 
enough  Information  to  locate  the  dataset. 

(c)  Each  user  is  assigned  a preferred  disk  volume  onto 
which  he  should  place  his  permanent  datasets.  An 
Indirect  reference  technique  Is  used  so  that  the  user 
does  not  need  to  know  the  serial  number  of  the  volume 
on  which  his  data  Is  stored.  The  user  supplies  the 
name  of  a dataset  known  to  be  on  his  preferred  volume 
and  requests  that  the  new  dataset  reside  on  the  same 
one.  The  dataset  referenced  is  a dummy  one  with  a 
name  of  the  form  userid. VOL  which  is  placed  on  the 
preferred  volume  at  the  time  the  user  is  authorized 
to  use  the  computer  system. 

This  technique  creates  a degree  of  volume 
Independence  and  helps  in  load  balancing,  which  will 
be  discussed  later. 
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3.2  Maintenance  procedures 

This  section  describes  several  disk  maintenance 

rocedures  In  use  at  W.R.E.  These  are  also  automated,  to 

Inlmlze  human  Involvement.  Reference  11  contains  detailed 
1 nformat I on . 

(a)  The  first  procedure  enforces  two  of  the  conventions 
described  above  - dataset  naming  and  cataloguing. 
The  procedure  is  performed  dally  and  deletes  any 
datasets  that  are  not  catalogued  or  that  do  not  begin 
with  a registered  userid.  Various  other  checks  are 
performed  to  ensure  that  the  disk  contents  and  the 
operating  system  catalogue  are  consistent.  This  Is 
called  the  clean-up  procedure. 

(b)  The  partitioned  dataset  organization  has  already  been 
mentioned.  Basically  this  allows  a single  dataset  on 
disk  storage  to  be  used  for  several  distinct  sets  of 
Information,  called  members  (typically  program 
modules).  A partitioned  dataset  (PDS)  contains  a 
directory,  or  Index,  which  Is  used  to  locate  each  of 
the  members.  Whenever  a member  is  replaced  by  an 
updated  version  It  Is  stored  at  the  end  of  the 
dataset  and  the  directory  Is  updated  to  reflect  the 
new  location.  Because  of  a limitation  In  the  design 
of  the  operation  system  the  space  originally  occupied 
by  the  member  Is  not  available  for  reuse.  The  size 
of  the  dataset  therefore  grows  as  members  are 
replaced . 

A technique  known  as  compressing  is  Included  in 
the  standard  software  to  reclaim  such  waste  space. 
The  disadvantage  with  this  technique  is  that  the 
compression  would  need  specific  user  action  and  the 
reclaimed  space  would  still  be  assigned  to  the 
dataset . 

By  building  upon  this  facility  a procedure  has 
been  developed  to  automatically  compress  all 
partitioned  datasets  updated  since  the  last  run,  and 
to  release  unused  space.  (The  date  of  the  last 
update  Is  available  as  a byproduct  of  the  migration 
system  - see  Section  4.1).  The  advantage  of  this 
process  is  the  extra  free  disk  space  It  provides 
without  user  Involvement.  However,  the  procedure 
also  has  the  secondary  benefit  of  Identifying  corrupt 
partitioned  datasets.  The  operating  system  allows  a 
partitioned  dataset  to  be  used  as  a sequential  output 
dataset,  thereby  destroying  Its  directory,  with  no 
Indication  of  this  having  happened.  This  had  caused 
problems  In  the  data  migration  software,  as  it  would 
with  any  attempt  to  use  the  dataset  as  a partitioned 
dataset.  The  compress  procedure  Is  arranged  so  that 
failure  to  compress  a single  dataset  does  not  affect 
the  rest  of  the  job.  A scan  of  the  output  Indicates 
any  offending  dataset  so  that  the  owner  can  be 
consulted.  This  problem  Is  expected  to  diminish  as 
users  become  more  familiar  with  dataset 
organ  I zat Ions . 


(c)  The  high  level 


of  dataset;  creation  and 


deletion 
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activity  causes  any  free  disk  space  to  be  quickly 
fragmented,  greatly  reducing  Its  usefulness.  This 
slows  the  allocation  of  new  datasets  by  the  operating 
system.  We  have  recently  acquired  a program  (ref. 
14)  that  reorganizes  a disk  volume,  collecting  all 
free  space  Into  a few  large  areas.  All  datasets  on 
the  volume  are  compressed  Into  one  contiguous  area 
(or  extent)  and  their  unused  space  released. 

From  the  results  obtained  so  far  It  Is  apparent 
that  the  degradation  of  the  reorganized  volumes 
occurs  fairly  rapidly  - within  +3  to  4 weeks.  We 
therefore  Intend  to  reorganize  every  3 weeks  to 
control  the  effects  of  fragmentation. 


4.  WEEKLY  MIGRATION 

Automatic  data  migration  Is  performed  weekly.  In  two  stages. 
In  the  first  stage  datasets  are  selected  for  migration  based  on 
the  current  and  projected  disk  storage  levels.  Warning  notices 
are  generated  and  sent  to  the  users  responsible  for  these 
datasets.  In  the  second  stage,  normally  run  two  days  later,  the 
actual  migration  takes  place. 

The  only  reason  for  the  two  stage  approach  Is  to  enable  the 
warning  notices  to  be  distributed  to  the  users.  Since  retrieval 
from  the  archives  is  not  automatic,  but  requires  user  action,  it 
is  necessary  to  establish  for  the  users  a firm  time  when  the 
selected  datasets  will  cease  to  be  available  on  disk.  This 
minimises  the  inconvenience  and  uncertainty  to  users. 

The  most  important  characteristic  of  a dataset  that  influences 
selection  for  migration  Is  Its  date  of  last  access.  Before 
describing  the  two  stages  of  the  weekly  process  in  more  detail  an 
explanation  Is  given  of  how  this  date  is  determined. 

4.1  Date  of  last  access 

The  operating  system  does  not  maintain  a last  access  date 
as  part  of  the  dataset  status  information,  either  on  disk  or 
in  the  catalogue. 

The  first  approach  considered  was  a modification  to  the 
operating  system  software  to  record  the  current  date  in  an 
unused  field  of  the  dataset  entry  In  the  disk  pack  directory 
(VTOC)  whenever  the  dataset  Is  accessed.  While  this  has  the 
advantage  that  the  access  date  would  be  maintained 
automatically  by  the  operating  system  it  would  involve  an 
extra  disk  access  in  a number  of  cases.  However  the  reason 
for  abandoning  this  approach  was  our  reluctance  to  rely  upon 
compatibility  of  future  releases  of  the  operating  system. 
An  alternative  scheme,  using  accounting  data  collected  by 
the  System  Management  Facilities  (SMF)  software(ref .6),  was 
therefore  adopted.  This  software  is  part  of  the  operating 
system  that  monitors  and  records  selected  activities  and 
events,  including  dataset  accesses.  This  data  is  intended 
for  accounting  but  also  aids  system  tuning. 

A set  of  programs  was  written  to  extract  the  dataset 
access  records  (types  14,  15  and  user  type  140)  from  the  SMF 
data  and  maintain  a file  of  all  dataset  names  currently  on 


T 


5 


WRE-TR-l8tO(A) 


disk/  the  date  of  their  last  access  and  whether  they  were 
altered  since  the  last  update  of  the  file.  This  Is  done 
immediately  before  running  the  first  stage  of  the  migration 

• system,  so  that  accurate  access  Information  Is  available  to 

It. 

The  operating  system  does  not  generate  a normal  access 
, record  (type  14)  for  the  second  and  subsequent  datasets  in  a 

concatenation  of  partitioned  datasets.  To  ensure  that  this 
data  is  recorded  an  SMF  user  record  (type  140)  is  generated 
in  an  SMF  exit  (lEFUJV)  for  all  datasets  appearing  in  a 
concatenation  (except  the  first).  The  format  of  type  140 
records  is  given  in  Appendix  IV. 

4.2  Stage  1 - Selection  of  datasets  for  migration 

The  first  task  of  Stage  1 is  to  calculate  the  amount  of 
free  space  that  must  be  released  by  migration,  if  any.  This 
is  determined  by  comparing  the  current  total  free  space  on 
all  disks  with  the  required  total  that  was  specified  in  the 
input  parameters. 

Once  the  amount  of  space  to  be  released  is  known  the 
selection  of  datasets  for  migration  can  proceed.  Users  may 
indicate  the  names  of  datasets  that  they  want  to  be  migrated 
(see  Section  5.6).  If  any  of  these  requests  have  been  made 
those  datasets  are  selected  first.  Next  the  program  applies 
‘ two  criteria  of  its  own  to  select  datasets.  Both  are 

controlled  by  input  parameters. 

• (i)  All  datasets  that  have  not  been  accessed  for  a 

specified  number  of  weeks  are  selected,  regardless 
of  the  amount  of  disk  space  to  be  reclaimed.  This 
is  generally  set  at  4 to  8 weeks  since  most  datasets 
not  accessed  for  this  length  of  time  are  probably  no 
longer  required.  However  users  may  have  been 
reluctant  or  not  sufficiently  conscientious  to 
delete  them,  so  that  migration  offers  a good 
compromi se . 

(li)  If  requests  by  users  for  migration  and  the  first 
selection  criterion  fail  to  produce  enough  free 
space  a secondary  selection  technique  is  applied. 
First  an  occupancy  figure  is  computed  for  all 

remaining  datasets.  This  is  done  by  multiplying  its 
size,  in  tracks,  by  the  number  of  days  since  it  was 

• last  accessed  - a calculation  that  can  easily  be 

adjusted  within  the  program.  Based  on  this  figure 
the  datasets  are  grouped  in  ranges  of  values  input 

. to  the  program.  All  datasets  falling  in  the  highest 

valued  group  are  then  selected  for  migration.  If 
enough  space  has  still  not  been  released 

successively  lower  valued  groups  are  selected  until 
no  further  space  Is  required. 

As  a consequence  of  Stage  1 each  user  responsible  for  any 
of  the  selected  datasets  receives  a report  warning  him  that 
they  are  to  be  migrated.  The  names  of  the  datasets  and  the 

time  the  migration  Is  to  occur  are  indicated  on  the  report. 

This  warning  gives  the  user  time  to  review  whether  the  data 
set  is  still  needed.  If  it  is  not  it  should  be  deleted. 

All  Stage  1 input  parameters  (including  the  desired  free 
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space  level  and  the  constants  defining  the  primary  and 
secondary  selection  criteria)  have  values  that  are  used  by 
default  If  they  are  not  overridden.  An  additional  form  of 
input,  via  a dataset,  may  be  used  to  identify  datasets  that 
are  to  be  kept  on  disk.  Together  these  features  provide  a 
flexible  tool  for  automatic  dataset  selection. 

As  a final  comment  on  Stage  1 input  parameters,  it  is 
observed  that  the  free  space  level  indicated  is  never 
actually  attained  although  enough  datasets  are  selected  to 
achieve  this  level  should  the  migration  be  done  immediately. 

In  practice  a slightly  higher  level  than  required  should  be 
specified  to  offset  the  additional  space  that  will  be  used 
prior  to  the  execution  of  Stage  2. 

4.5  Stage  2 - Migration  of  the  selected  datasets 

Stage  2 of  the  migration  scheme  transfers  to  magnetic 
tape  those  selected  datasets  that  still  remain  on  disk  as 
well  as  any  datasets  flagged  for  migration  by  users  (see 
Section  5.6)  since  Stage  1 was  executed.  Disk  space  is 
freed  as  soon  as  the  datasets  are  copied  onto  tape 
successfully.  The  serial  number  of  the  tape  to  be  used  is 
part  of  the  program  input.  The  standard  software  permits 
multiple  datasets  to  be  written  onto  a tape  and  these  are 
assigned  a sequence  number  starting  at  one.  The  first 
dataset  on  each  tape  is  a dummy  with  a name  of  the  form 
SYSl . XXnnnnnn  where  nnnnnn  is  the  tape  serial  number.  The 
migrated  datasets  therefore  begin  at  sequence  number  two. 

Since  some  of  these  datasets  may  be  password-protected  it 
is  imperative  that  their  security  be  maintained  on  the 
archive  tapes.  Because  the  operating  system  requires  that 
all  datasets  on  a single  tape  volume  have  the  same 
protection  level  each  migrated  dataset  must  be  assigned 
read/write  protection.  The  way  In  which  this  conflict  is 
resolved  will  be  discussed  later  under  dataset  retrieval 
{ Sec  t i on  5.1). 

The  programs  that  comprise  Stage  2 of  the  automatic 
migration  system  have  privileged  status  so  that  they  are  not 
required  to  know  the  passwords  of  protected  datasets  in 
order  to  transfer  them  to  tape.  This  feature  is  not 
available  to  user  programs. 

While  a dataset  is  in  the  archives  it  has  an  expiry  date 
associated  with  it.  At  the  end  of  the  week  in  which  this 
date  falls  the  dataset  is  deleted,  unless  action  is  taken  by 
the  user  beforehand.  The  options  available  to  the  user  will 
be  discussed  later.  Datasets  automatically  migrated  (i.e. 
without  user  request)  have  an  initial  retention  period  ofj  3 
months . 

Finally  Stage  2 generates  a report  indicating  the  current 
contents  of  the  archives  and  the  additions  made  by  this  run. 

In  addition  a separate  report  Is  sent  to  each  user  who 

(a)  has  had  datasets  migrated  this  week,  or 

(b)  has  datasets  with  less  than  one  month  until  expiry,  or 

(c)  has  had  datasets  expire  during  the  week. 

The  report  lists  the  names  and  expiry  dates  of  all  the 
user's  datasets  currently  in  the  archives.  Those  due  to 
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expire  during  the  coming  month  are  flagged. 

Whenever  a dataset  is  migrated  it  replaces  any  dataset  of 
the  same  name  that  might  already  exist  on  archive  tapes, 
t Should  this  happen  the  original  is  only  available  by  special 

arrangement.  The  user  report  will  indicate  any  such 

occurrence,  to  draw  attention  to  the  fact  that  access  to 
. data  may  have  been  lost. 

4.4  Balancing  of  disk  loads 

As  mentioned  in  Section  4.2  Stage  1 acquires  a certain 
total  amount  of  free  space,  regardless  of  its  distribution. 
An  alternate  approach  that  would  ensure  that  all  disks  had 
the  same  free  space  level  was  also  considered.  This  could 
be  done  by  applying  the  selection  criteria  to  each  disk 
separately.  Although  this  at  first  appears  to  be  a 
reasonable  approach  it  has  the  disadvantage  that  users  are 
not  treated  equally.  Those  assigned  to  a heavily  loaded 
disk  would  be  subject  to  more  severe  selection  criteria  than 
those  on  a lightly  loaded  one  and  this  was  considered  an 
undesirable  situation.  Although  the  approach  used  tends  to 
create  load  imbalances,  it  does  treat  users  impartially. 

A scheme  was  devised  to  help  reduce  any  imbalance  created 
either  by  the  migration  scheme  or  by  injudicious  assignments 
of  users  to  disk  packs.  Each  week,  immediately  after 
running  Stage  2,  the  free  space  levels  on  each  disk  are 
compared.  If  there  are  no  serious  anomalies  no  action  is 
taken.  However  if  there  is  a disk  with  significantly  less 
• free  space  than  the  average  the  imbalance  is  corrected.  A 

report  program  lists  the  space  each  user  occupies  on  each 
disk.  From  these  figures  enough  users  can  be  selected  for 
transfer  to  more  lightly  loaded  disks.  The  selections  are 
input  to  a program  which  generates  utility  program  control 
statements  to  perform  the  routine  dataset  transfers  and 
update  the  operating  system  catalogue.  Datasets  that  are 
not  sequential  or  partitioned  are  not  moved  automatically, 
but  are  reported  for  investigation.  In  most  cases  the  users 
need  not  be  informed  of  their  transfer  because  of  the  high 
degree  of  volume  independence  created  by  the  conventions 
described  in  Section  3.1. 


5.  USER  FACILITIES  | 

Users  have  little  control  over  the  automatic  weekly  migration  i 

procedures  described  in  the  previous  section.  However  special 
features  are  provided  for  users  to  manage  their  datasets  in  the 
archives  and  to  use  the  system  to  their  own  advantage.  This 
section  discusses  these  features,  which  may  be  invoked  either 
from  a batch  job  by  using  a catalogued  procedure  or  from  a TSO 
terminal  by  using  a command  procedure.  The  syntax  will  not  be  i 

given  here.  The  WRE  Central  Computer  User's  Gu i de ( ref . 10 ) 
provides  this  information.  j 

5.1  Retrieving  datasets  from  the  archives  j 

Before  a dataset  in  the  archives  can  be  reused  it  must  be  | 

returned  to  the  user's  preferred  disk  volume.  A retrieval  | 

capability  is  therefore  a key  requirement.  Two  similar  ] 
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procedures  (RETRIEVE  and  RELOAD)  are  provided  to  perform 
this  function.  The  only  difference  between  the  two  Is  that 
RELOAD  leaves  a copy  of  the  dataset  In  the  archives  on 
completion,  whereas  RETRIEVE  does  not.  The  only  parameters 
required  are  the  dataset  name  and  optionally  a password, 
which  must  be  the  control  password  of  the  dataset. 

The  steps  performed  by  the  software  to  retrieve  a dataset 
are  - 

(I)  A check  is  first  made  to  see  If  the  dataset  is 
actually  In  the  archives.  If  so.  Its  protection 
status  and  location  (I.e.  the  archive  tape  serial 
number  and  the  dataset  sequence  number)  are 
determined.  If  not,  the  request  of  course  falls. 

(II)  If  the  dataset  Is  password-protected  and  the  user 
did  not  supply  a password,  or  supplied  an  Incorrect 
one  (i.e.  not  the  control  password)  the  retrieval 
falls. 

(Ill)  If  the  dataset  is  not  protected  (I.e.  has  no  entry 
In  the  PASSWORD  dataset),  then  a dummy  password 
must  be  created  for  it.  If  this  were  not  done  any 
attempt  at  copying  the  dataset  to  disk  would  fail, 
since  the  protection  indicators  are  set  In  the  tape 
label  (see  (v)  below). 

(Iv)  The  disk  volume  to  receive  the  dataset  Is 
Identified.  If  the  dataset  already  exists  on  this 
volume  then  it  Is  first  deleted.  Similarly  If  the 
dataset  Is  already  catalogued  then  the  catalogue 
entry  is  also  deleted.  Next,  if  the  dataset  Is 
sequential,  space  is  pre-al located  on  the  receiving 
volume.  Sequential  datasets  will  have  a primary 
allocation  equal  to  the  total  space  occupied  by  the 
dataset  before  being  migrated  (In  tracks)  and  a 
secondary  allocation  of  20?i  of  this  amount. 
Partitioned  datasets  will  retain  the  primary  and 
secondary  allocations  held  prior  to  being  migrated. 

(v)  The  archive  tape  Is  mounted  and  the  dataset  Is 
copied  to  disk.  The  operating  system  checks  the 
password  at  this  stage. 

(vi)  If  the  dataset  is  not  protected  the  dummy  password 
created  in  (III)  above  Is  removed. 

(vli)  Upon  successful  transfer  the  dataset  Is  catalogued 
In  the  system  catalogue.  If  the  dataset  was 

password  protected  and  has  already  been  allocated 
to  another  batch  job  or  TSO  session,  or  Is  expiry 
date  protected,  then  it  will  not  be  protected  on 
disk.  A warning  message  Is  issued  when  this 
happens.  If  the  RETRIEVE  procedure  was  used  the 
dataset  will  also  be  removed  from  the  archives. 
However,  with  RELOAD,  It  will  remain  In  the 

a rch  I ves  . 

Once  returned  to  disk  the  dataset  bears  no  marks  of 
having  been  In  the  archives.  It  Is  treated  In  the  same  way 
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as  any  other  dataset  and  may  again  be  selected  for  migration 
at  a later  date.  However  the  actual  retrieval  is  regarded 
as  a dataset  access,  so  that  It  should  not  be  migrated  again 
for  some  time. 

5.2  Scratching  datasets  from  the  archives 

A procedure  (ASCRATCH)  for  deleting  a dataset  from  the 
archives  Is  provided.  This  should  be  used  whenever  the  user 
is  certain  that  the  dataset  Is  no  longer  required,  rather 
than  wait  for  its  expiry  date  to  pass.  If  the  dataset  is 
protected  the  correct  password  must  be  supplied  - otherwise 
the  dataset  will  not  be  scratched. 

5.3  Increasing  the  expiry  date 

As  mentioned  previously  all  datasets  In  the  archives  have 
an  associated  expiry  date.  Users  are  warned  when  the  expiry 
date  of  one  of  their  datasets  is  approaching.  They  may 
disregard  the  warning  and  allow  the  dataset  to  be  removed 
from  the  archives. 

Alternatively,  if  the  dataset  Is  still  required,  the  user 
may  invoke  a procedure  (EXPIRY)  to  extend  the  expiry  date. 
In  fact  this  may  be  done  at  any  time.  Unless  specified 
otherwise  the  procedure  applies  an  extension  of  6 months. 
The  maximum  retention  period  of  any  dataset  is  12  months  so 
that  users  must  review  their  need  for  the  dataset  at  least 
once  a year.  No  password  is  required  to  Increase  the  expiry 
date . 

5.4  Archiving  on  demand 

Users  have  the  ability  to  archive  their  own  datasets  on 
demand  (procedure  ARCHIVE).  A standard  retention  period  of 
6 months  is  applied  to  these  datasets  unless  otherwise 
specified.  However  the  12  month  maximum  is  still  enforced. 
The  dataset  must  be  d I sk- res ident  and  catalogued  and  its 
organization  must  be  one  of  those  currently  handled  by  the 
system  (sequential  or  partitioned).  In  addition,  if  the 
dataset  is  password  protected,  its  control  password  must 
also  be  supplied.  If  any  of  these  conditions  are  violated 
the  request  to  archive  will  be  denied. 

The  steps  performed  by  the  software  to  archive  a dataset 
are : 

(I)  The  dataset  is  located  by  using  the  catalogue  and 
its  attributes  are  obtained  from  the  disk's 

di rectory. 

(li)  If  this  dataset  Is  protected  and  the  user  did  not 
supply  a password,  or  supplied  an  incorrect  one, 
the  request  fails. 

(iii)  The  archive  tape  is  loaded  and  the  dataset  is 
transferred  to  it,  following  all  existing  data. 
The  program  that  performs  this  data  transfer  does 
ail  the  password  checking  required  (see  (li) 
above).  It  runs  as  a privileged  program  with  the 
bypass-password-protection  feature  to  circumvent 
the  operating  system's  check  on  the  password  of  the 
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previous  dataset  on  the  archive  tape,  which  is  not 
known  to  the  software  at  this  time. 


(vi)  Upon  successful  transfer  the 
from  both  disk  and  catalogue. 


dataset 


deleted 


The  current  archive  tape  serial  number  and  the  number  of 
datasets  on  it  are  obtained  from  a dataset  originally 
created  by  Stage  2 of  the  weekly  migration  process.  If  the 
archival  was  successful  this  information  is  updated  in 
readiness  for  the  next  request. 

Note  that/  as  in  Stage  2,  the  dataset  will  replace  one  of 
the  same  name  that  might  already  be  In  the  archives.  An 
output  message  will  indicate  when  this  happens. 

5.5  Backing-up  disk  datasets 

There  is  often  a requirement  for  creating  a back-up  copy 
of  a disk  dataset.  For  Instance,  this  might  be  done  to 
guard  against  accidental  loss  or  damage  to  a dataset  that 
would  be  difficult  to  recreate.  The  user  might  also  want  to 
back-up  the  current  working  version  of  his  dataset  before 
altering  it.  This  capability  Is  provided  by  using  the 
archives  to  store  the  back-up  copy  (procedure  BACKUP).  The 
default  retention  period  of  6 months  and  maximum  of  12  also 
apply  here.  In  fact  the  operation  of  this  procedure  is  the 
same  as  the  ARCHIVE  procedure  described  in  Section  5.4, 
except  that  the  dataset  is  not  deleted  from  the  disk  or  the 
catalogue  at  completion.  The  back-up  copy  is  treated  the 
same  as  any  other  dataset  in  the  archives.  It  will  appear 
on  the  reports  sent  to  users  and  can  be  retrieved,  scratched 
etc.  The  expiry  date  of  course  applies  only  to  the  copy, 
not  to  the  original  that  is  still  on  disk. 

5.6  Migrating  disk  datasets 

In  Section  5.4  a method  for  users  to  voluntarily  archive 
datasets  on  demand  was  described.  While  this  is  to  be 
encouraged  when  free  disk  storage  space  is  at  a premium  it 
has  one  disadvantage  - the  operational  overhead  of  a tape 
mount  incurred  because  of  the  Immediate  transfer  of  the 
data.  An  alternative  method  requiring  no  operational 
involvement  is  available  when  immediate  archival  is  not 
necessary.  This  procedure,  called  MIGRATE,  simply  flags  the 
dataset,  indicating  that  it  is  to  be  migrated  by  the 
automatic  migration  process  at  the  end  of  the  week.  The 
dataset  will  therefore  remain  on  disk  for  up  to  a week  after 
it  has  been  flagged.  There  is  no  restriction  to  accessing 
the  dataset  during  this  period.  A procedure  called  GMIGRATE 
is  also  provided  to  flag  a relative  generation  of  a 
generation  data  group  (GDG)  for  migration.  The  standard 
default  of  6 months  and  maximum  of  12  months  for  the 
retention  period  also  apply  to  these  procedures. 

5.7  Listing  the  names  of  datasets  in  the  archives 

Although  users  periodically  receive  a report  indicating 
the  status  of  their  datasets  that  are  currently  in  the 
archives  (see  Section  4.3)  there  will  be  occasions  when  an 
up-to-date  list  is  required  on  request,  e.g.  to  verify  the 
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result  of  invoking  one  of  the  features  mentioned  above.  The 
software  provides  this  facility  with  a procedure  called 
LI  STARCH. 


6.  ARCHIVE  TAPE  MAINTENANCE 

The  amount  of  live  information  on  an  archive  tape  gradually 
decreases  as  the  result  of  retrievals  and  deletions.  With  at 
least  one  new  tape  created  each  week  the  number  could  soon  become 
excessive.  A scheme  was  developed  to  reduce  the  number  of  tapes 
and  make  more  efficient  use  of  them  by  a process  of  selective 
transcription. 

A report  program  lists  the  live  datasets  on  each  archive  tape 
and  the  total  disk  space  represented  by  them.  From  this  list 
tapes  are  selected  for  transcription.  The  serial  numbers  of  the 
selected  tapes,  together  with  those  of  one  or  more  new  output 
tapes,  are  input  to  an  automatic  transcription  program.  The  only 
other  parameters  supplied  define  the  maximum  usage  level  of  each 
output  tape.  This  can  be  specified  as  the  maximum  number  of 
datasets  it  is  to  contain  and/or  the  maximum  space  that  they  may 
represent.  The  program  transfers  all  live  datasets  from  the 
input  tapes,  mounted  in  turn,  to  the  output  tapes.  When  an 
output  tape  has  reached  the  maximum  usage  level  it  is  unloaded, 
another  mounted  and  the  transcription  continued. 

Archive  tapes  with  no  live  datasets  (which  includes  input 
tapes  after  transcription)  are  retained  for  at  least  one  month 
for  the  convenience  of  users  who  may  wish  to  reclaim  lost  data. 
After  this  period  the  tapes  are  released  to  the  general  tape 
1 ibrary  for  reuse. 

Duplicate  copies  of  all  archive  tapes  are  also  kept  for  the 
life  of  the  original  in  case  of  damage  or  loss.  Reference  4 
examines  the  subject  of  the  lifetime  expectancies  of  storage 
media. 


7.  EMERGENCY  PROCEDURES 

An  emergency  condition  is  considered  to  exist  if,  during  the 
normal  operation  of  the  computer,  a user  disk  becomes  full.  This 
is  of  course  an  intolerable  situation  and  immediate  action  must 

be  taken  to  raise  the  free  space  on  the  disk  to  an  acceptable 

level.  The  following  steps  may  be  taken. 

(i)  Selected  datasets  are  transferred  to  more  lightly  loaded 
disks  and  datasets  that  have  been  flagged  for  migration 
by  users  are  migrated  immediately. 

(ii)  The  disk  load  balancing  software  may  be  used  to  transfer 
one  or  more  users  off  the  offending  disk,  provided  that 
there  are  other  volumes  with  enough  free  space  to  absorb 
the  extra  datasets  without  themselves  becoming 

Intolerabl y full. 

(Ill)  Failure  to  solve  the  problem  by  the  above  two  techniques 
results  in  more  direct  action.  Datasets  with  large 

occupancy  figures  are  selected  from  the  offending  disk 
for  Immediate  archival,  after  consultation  with  their 
owners . 
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8.  SOFTWARE  IMPLEMENTATION 
8.1  Archive  catalogue 

The  archive  catalogue  is  the  focal  point  of  the  software 
implementation.  This  is  a key-sequenced  (VSAM)  dataset  that 
reflects  the  current  state  of  the  archives.  It  contains  one 
80-byte  record  for  each  dataset/  with  the  44-byte  name  as 
the  key. 

The  Information  contained  in  each  record  Is 

- the  dataset  name 

- the  dataset  type  (sequential  or  partitioned) 

- the  dataset  protection  Indicators 

- the  date  cf  last  access 

- the  date  of  archival  or  migration 

- the  exp  I ry  date 

- the  disk  space  originally  occupied 

- the  disk  volume  currently  associated  with  the  dataset. 
This  also  indicates  the  volume  to  which  It  will  be 
returned  on  retrleval.lt  is  initially  the  volume 

from  which  the  dataset  was  removed,  but  may  be 
changed  by  the  load  balancing  software. 

- the  serial  number  of  the  archive  tape  on  which  the 
dataset  now  resides 

- the  dataset  sequence  number  on  the  archive  tape. 

These  records  are  created  as  the  result  of  an  automatic 
migration  or  user  demand  archival  or  backup  and  are  deleted 
as  the  result  of  a dataset  being  deleted  from  the  archives, 
its  expiry  date  lapsing  or  a dataset  being  replaced  by  one 
of  the  same  name.  There  are  also  several  processes  that 
amend  a record.  These  are  a change  to  the  expiry  date,  disk 
load  balancing  techniques  (when  the  disk  volume  entry  for 
each  of  the  user's  datasets  is  updated),  and  archive  tape 
transcription  (when  the  last  two  entries  are  updated). 

The  removal  of  a dataset  from  the  archives,  (e.g.  by  a 
replacement)  involves  only  a change  to  the  archive 
catalogue,  and  no  physical  tape  activity.  The  dataset  is 
therefore  still  on  tape,  giving  protection  in  case  of  user 
error . 

Obviously  the  destruction  of  the  archive  catalogue  would 
seriously  disrupt  the  operation  of  the  system,  unless  some 
precautions  were  taken.  Should  damage  occur  it  is  necessary 
to  restore  the  catalogue  to  its  original  state  as  quickly 
and  with  as  little  inconvenience  as  possible.  This  requires 
some  form  of  event  recording  by  the  software.  The  System 
Management  Facilities  (SMF),  mentioned  in  Section  4.1, 
provide  a useful  tool  for  this  purpose.  In  that  authorized 
programs  are  permitted  to  write  records  Into  the  SMF 
dataset.  Therefore  whenever  an  alteration  is  made  to  the 
archive  catalogue  an  SMF  user  record  is  generated  indicating 
the  type  of  alteration  (addition,  deletion  or  amendment)  and 
the  contents  of  the  catalogue  entry.  For  an  amendment  the 
record  contains  the  new  contents  of  the  catalogue  entry. 

A program  was  written  to  reconstruct  the  archive 
catalogue.  The  input  to  the  program  is  an  old  copy  of  the 
catalogue,  the  date  and  time  the  copy  was  made,  and  the  SMF 
data  generated  since  that  time.  The  program  extracts  the 
alterations  In  sequence  from  the  SMF  data  and  reapplies  them 


e<ich)  were  available  for  user  data.  Of  these  one  was  used  for  a 
time  solely  by  ADP  Section  and  was  not  included  in  the  automatic 
weekly  migration  scheme. 

The  first  datasets  were  migrated  on  4/11/75,  when  the  average 
free  space  on  the  two  disks  first  fell  below  25%.  This  Is  the 
value  we  anticipated  would  permit  a week's  operation  without 
emergencies.  For  the  following  few  weeks,  while  the  system  was 
settling  in,  no  meaningful  usage  statistics  could  be  gathered. 
However,  after  this  period,  the  system  performed  very  well  with 
the  25%  free  space  level.  This  situation  prevailed  until  the  end 
of  Fetaruary  1976,  when  the  computer  was  transferred  to  its 
p<?rmanent  location. 

During  March  1976  there  was  very  little  activity  on  the 
computer  and  no  migration  was  done.  From  the  beginning  of  April 
six  disk  packs  were  available  for  user  data  and  migration  was 
p«‘rformed  only  irregularly  during  the  next  six  weeks.  During 
this  period,  with  the  increased  availability  of  the  computer,  the 
amount  of  permanent  user  data  grew  rapidly,  and  in  fact  continues 
to  do  so.  In  an  endeavour  to  hold  some  capacity  in  reserve  the 
migration  scheme  was  set  to  maintain  about  40%  of  free  space  from 
the  beginning  of  May.  As  the  demand  for  space  grew  this  figure 
was  gradually  reduced,  and  is  currently  being  held  at  30%. 

The  results  for  the  first  period  (17/11/75  to  27/2/76)  are 
compared  with  those  for  a second  (1/4/76  to  27/7/76)  and  a third 
(5/9/76  to  7/12/76)  in  Table  1.  The  figures  reveal  the 
increasing  use  of,  and  requirement  for,  the  migration  system. 
Note  that  currently  the  average  size  of  the  datasets  in  the 
archives  is  about  250,000  bytes,  or  just  over  a cylinder. 


10.  SUMMARY 

The  features  of  an  automatic  data  migration  system  as  it 
currently  exists  have  been  described.  The  operational  results 
obtained  so  far  have  validated  our  decision  to  develop  the 
system.  Without  it  our  disk  space  would  surely  be  unmanageable. 
The  number  of  retrievals  has  not  been  excessive  and  the  average 
time  to  honour  a request  is  only  five  minutes.  In  addition  the 

migration  selection  criteria  have  been  quite  successful  In 

maintaining  adequate  free  space  levels,  as  evidenced  by  the  very 
few  emergency  situations  that  have  occurred.  Even  the  disk  loads 
have  remained  reasonably  well  balanced,  with  user  transfers  being 
required  only  every  month  or  so.  We  had  expected  much  more 

frequent  Imbalances  than  this. 

Development  in  the  near  future  will  concentrate  on  the  known 
limitations  of  the  system,  as  follows  - 

(i)  Replacement  of  the  IBM  utility  program  I EHMOVE  as  the 
dataset  transfer  tool.  A faster  program  with  the  same 
versatility  as  I EHMOVE  would  be  preferred. 

(ii)  Reassessment  of  the  need  to  include  other  dataset 

organisations  in  the  system,  particularly  the  VSAM 
organisation,  which  is  used  for  indexed  sequential 
applications.  However  suitable  techniques  for  handling 
VSAM  datasets  must  first  be  developed. 


( i i i ) Further 


investigation  of  techniques  for  handling 
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datasets  used  on  a cyclic  basis. 

(Iv)  Modification  of  the  migration  and  user  demand  archival 
software  to  automatically  switch  to  a new  archive  tape 
when  the  current  one  becomes  full. 

(v)  Provision  of  a delayed  retrieval  capability  to  be  run  at 
night.  Non-urgent  requests  could  be  batched-up  and  run 
as  a single  job,  greatly  reducing  tape  handling  during 
prime  operations  time. 

(vl)  Modifications  of  Stage  2 of  the  automatic  migration 
process  to  write  to  more  than  one  tape  simultaneously. 
This  should  reduce  the  elapsed  time  considerably. 

In  addition  it  would  be  desirable  to  make  dataset  retrievals 
automatic.  If  this  were  possible  the  weekly  cycle  could  be 
single  stage,  since  users  would  not  require  warning  notices.  In 
fact  the  whole  migration  system  could  be  made  transparent  to  the 
user.  Unfortunately  all  investigations  In  this  area  have  failed 
to  reveal  a satisfactory  solution  to  the  problem,  and  require 
modifications  to  the  operating  system  code  which  could  not  be 
made  through  standard  interfaces. 
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APPENDIX  I 

MIGRATION  SYSTEM  DATASETS 

The  format  and  contents  of  the  various  datasets  used  by  the 
migration  software  are  described  in  this  appendix. 

I .1  SYSl. ARCHIVE. CATLG 

This  is  a VSAM  key-sequenced  dataset  with  one  80-byte 
record  for  each  dataset  in  the  archives.  The  key  is  the 
44-byte  dataset  name.  A backup  copy  of  the  dataset,  called 
SYSl. ARCHIVE. COPYCAT,  exists  on  a different  disk  volume. 


Offset 

S i ze 

Description 

0 

1 

flag  byte 

- b i t 0 : 0 i f dataset  i s 

parti tioned 
: 1 if  dataset  is 
sequent lal 

- bits  1-2  : dataset  protection 

bits  f rom  DSCB 

- bits  3-7  : unused  (zero) 

1 

44 

dataset  name 

45 

5 

last  access  date  (Julian  form 
i.e.  yyddd  - numeric, 
display  format) 

50 

5 

date  migrated  (Julian  - 

numeric,  display  format) 

55 

5 

expiry  date  (Julian  - numeric, 
display  format) 

60 

5 

dataset  size  in  tracks  - 

(numeric,  dlsolay  format) 

65 

6 

archive  tape  volume 

71 

3 

dataset  sequence  number  on 
archive  tape  (numeric,  display 

format ) 

74 

6 

disk  volume  to  be  used  for 
retrieval . 

I 1 
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1.2  SYSl. DATASET. LASTACC 


This  is  a VSAM  key-sequenced  dataset  with  one  50-byte 
record  for  each  dataset  on  a user  disk  volume.  The  key  is 
the  44-byte  dataset  name.  The  dataset  is  only  updated 
weekly,  during  Stage  1 of  the  automatic  migration  process.  A 
backup  copy  of  the  dataset,  called 
SYSl. DATASET. LASTACC. BACKUP,  resides  on  a different  disk 
vol ume . 

Offset  Size  Description 


0 


1 


1 44 

45  5 


flag  byte 

- bits  0-1  ; unused  (zero) 

- bits  2-5  : the  initial 
retention  period  of  the 
dataset  (in  months)  if  bit 
6 is  on.  If  bit  6 is  off 
these  bits  contain  zeros. 

- b i t 6 : i f on  then  the 
data  set  has  been  flagged 
for  migration  by  the  user. 

- bit  7 : i f on  then  the 
dataset  was  modified 
between  the  last  two 
updates  of  the  file 

dataset  name 

last  access  date  (Julian 

form  i.e.  yyddd  - numeric, 
display  format) 
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I .3  SYSl. ARCHIVE. PARMLIB 

This  is  a partitioned  dataset  containing  several  members, 
used  throughout  the  migration  system.  The  logical  record 
length  of  the  dataset  is  80  bytes.  A backup  copy  of  the 
dataset,  called  SYSl . ARCH  I VE . PARML I B. BACKUP,  resides  on  a 
different  disk  volume.  Both  must  be  compressed  from  time  to 
time  to  reclaim  waste  space  generated  when  members  are 
replaced . 

The  contents  of  the  more  important  members  are  described 


in  Appendix 
them.  These 

II  in  assoc i at i on 
are  - 

with  the  programs  that  use 

Member 

Used  In 

Descr i pt i on 

ARCHTAP 

ARCHIVE, F0RCE2 

The  current  archive  tape,  and 
the  next  unused  sequence 
numbe  r . 

DISKS 

VTOCDSN,USERSPC 

A list  of  the  disk  volume  serial 
numbers  whose  VTOCs  are  to  be 
read . 

LASTSMF 

SMF1415 

Time  stamp  of  the  last  SMF 
record  used  for  determining 
dataset  accesses. 

TRKDAYS 

WARNING 

A list  of  track-day  ranges 
used  in  dataset  selection. 

XMPTJOBS 

Dl RECUP 

Jobs  whose  dataset  accesses 
are  disregarded. 

XMPTSTEM 

WARNING, ARCHIVE 

Userids  and  dataset  names  exempt 
f rom  migration. 

The  remaining  members  contai 
util i ty  programs . 

in  control  statements  for  IBM 

WRi:-TK-  1870(A) 
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1.4  SYSl. ARCHIVE. WARNLI ST 

This  Is  a sequential  dataset  with  a logical  record  length 
of  80  bytes.  It  is  generated  and  used  in  Stage  1 of  the 
automatic  migration  run  and  passed  to  Stage  1,  which  deletes 
i t on  compl et i on . 

The  first  record  contains  the  date  the  datasets  will  be 
transferred  to  the  archives.  The  first  byte  of  this  record 
is  a blank  and  the  next  8 bytes  contain  the  date  In  the  form 
dd/mm/yy . 

The  remaining  records  indicate  the  datasets  that  have  been 
selected  for  migration.  They  are  in  ascending  order  of 
dataset  name,  with  the  following  format. 


Offset 

S i ze 

Desc  r i pt i on 

0 

44 

dataset  name 

44 

5 

creation  date  (Julian  form  - 
numeric,  display  format) 

49 

5 

last  access  date  (Julian  form 
numeric,  display  format) 

54 

6 

track-day  value  (or  occupancy 
figure)  - see  Section  4.2(ii) 

60 

20 

blank. 

1.5  SYSl. ARCHIVE. ARCHLIST 

This  is  a sequential  dataset  with  80-byte  logical  records. 
It  is  generated  and  used  solely  by  Stage  2 of  the  automatic 
migration  run.  However,  it  also  exists  between  runs  of  Stage 

2. 

The  first  record  contains  the  date  on  which  the  migration 
actually  occurred,  starting  in  byte  2,  in  the  form  dd/mm/yy. 

The  remaining  records  indicate  the  datasets  that  were 
migrated,  in  alphabetic  order. 

Offset  Size  Description 

0 44  dataset  name 

44  4 contains  'REPL'  if  the  dataset 

replaced  one  of  the  same  name 
already  in  the  archives.  Otherwise 
it  is  blank. 

48  5 expiry  date  (Julian  form  - numeric, 

display  format) 

53  6 disk  volume  from  which  the  dataset 

was  removed 

59  6 archive  tape  volume 

65  3 sequence  number  of  the  dataset  on 

the  archive  tape  (numeric,  display 
format ) 

68  12  blank. 

1 .6  SYSl. USERS 

This  is  a VSAM  key-sequenced  dataset  with  one  70-byte 
record  for  each  registered  userid.  The  key  is  the  userid, 
which  can  be  from  one  to  eight  bytes. 

The  dataset  is  used  for  the  automatic  addressing  of  the 
ices  generated  by  both  stages  of  the  weekly  migration 
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system.  It  has  similar  applications  In  the  automatic 
generation  of  labels  for  attaching  to  computer  bulletins  and 
other  notices  generated  by  Computing  Services  Group. 

The  dataset  has  very  low  update  activity  and 
reorganization  should  rarely  be  necessary. 

Offset  Size  Description 


0 

20 

20 

8 

28 

10 

38 

12 

50 

4 

54 

3 

57 

13 

1.7  SYSl. ARCHIVE. PLI 


User  name  - 
bytes  1-5  : Initials 
bytes  6-20  : surname 
user  I d 

abbreviation  of  user's  group 
building  in  which  user  works 
user's  phone  number 
user's  cost  centre 
space  for  expansion  (currently 
blank) . 


This  is  a partitioned  dataset  containing  the  source  code 
of  each  program  module  (including  the  assembler  language 
routines).  The  member  names  are  the  same  as  the  module 
names,  as  described  In  Appendix  II.  This  dataset  must  be 
compressed  occasionally,  depending  on  update  activity. 


I .8  SYSl. ARCHIVE. LOAD 

This  partitioned  dataset  contains  the  load  module  of  each 
routine  In  SYSl . ARCH f VE . PL  I , the  member  names  being  the  same 
as  those  of  the  source  modules.  Aliases  are  also  defined  for 
those  routines  with  alternate  entry  points.  Whenever  a 
source  module  Is  replaced  in  SYSl .ARCH  I VE . PL  I It  should  be 
compiled  and  the  load  module  replaced  In  this  dataset.  This 
ensures  that  the  contents  of  the  two  datasets  are  consistent. 
SYSl .ARCH I VE . LOAD  must  also  be  compressed  from  time  to  time. 


1.9  SYSl.ARCHI VE.CNTL 

This  partitioned  dataset  is  used  by  the  TSO  command 
procedures  described  in  Appendix  III.  Each  member  contains  a 
basic  set  of  JCL  that  the  command  procedure  for  each 
particular  function  edits  and  submits  as  a batch  job.  The 
members  are  RETRIEVE,  RELOAD,  ASCRATCH,  EXPIRY,  ARCHIVE  and 
BACKUP. 
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PROGRAM  MODULES 

In  this  Appendix  each  module  in  the  migration  scheme  is 
briefly  described.  These  are  all  contained  in  SYSl .ARCHI VE . PL  I 
and  their  load  modules  in  SYSl . ARCH  I VE . LOAD . The  type  of  module, 
the  programming  language  used,  and  the  names  of  associated 
modules  are  also  given.  The  subprograms  are  described  first. 

I I . 1 Subprograms 

The  description  of  each  subprogram  is  concluded  with  a 
list  of  the  arguments  used  in  its  calling  sequence.  The  PL/ I 
attributes  of  each  are  also  indicated.  Those  arguments 
followed  by  (S)  must  be  supplied  by  the  calling  program  - all 
others  are  returned  by  the  subprogram.  in  addition  all 
assembler  language  routines  must  be  declared  in  the  PL/ I 
calling  program  with  the  ASSEMBLER  option. 

(1)  ELAPSED 

type  - PL/1,  function  subprogram 

called  from  - EXP  I RY, F0RCE2 , LSTADSN, L I STDSN, 

WARNING, LSTARCH 

This  function  returns  the  number  of  days  between  two 
Julian  dates  passed  as  arguments, 
function  attributes  - FIXED  DECIMAL(3,0) 

Arguments  - 

first  date(S)  - PICTURE  9(5) 
second  date  (S)  - PICTURE  9(5) 


(2)  ENQDEQ 

entry  points  - ENQCAT, DEQCAT, ENQLACC, DEQLACC , ENQPARM, 
DEQPARM 

type  - assembler,  subroutine 

called  from  - EXP  I RY, SCRATCH, POSTDS, F0RCE2, LSTUSER, 

CHNGVOL,RETRVE, LSTADSN, LI STDSN, SHI  FT  ID, 
WARN  I NG, UPCAT, CHCKDSN, DUMPSMF 
This  routine  Issues  ENQ  and  DEQ  macros  for  three 
datasets  used  by  the  software  - the  archive  catalogue 
( SYSl . ARCH  1 VE . CATLG ) , the  last  access  dataset 
( SYSl . DATASET . LASTACC ) and  the  parameter  dataset 
(SYSl . ARCH  I VE . PARML I B) . These  datasets  are  always 
allocated  with  SHR  disposition  and  ENQ  is  used  to  obtain 
exclusive  use  for  any  read  or  write  access.  This 
minimizes  contention  problems. 

The  major  name  (qname)  and  minor  name  (rname)  for  each 
of  the  datasets  are  - 

SYSl. ARCHIVE. CATLG  - QCATARCH, RCATARCH 

SYSl. DATASET. LASTACC  - QLACCARC, RLACCARC 

SYSl. ARCHIVE. PARMLIB  - QPARMARC, RPARMARC 

Arguments  - none 

(3)  GETUSER 

type  - PL/ I , subroutine 
called  from  - ADDRESS 

This  routine  obtains  the  user  details  from  the  special 
dataset  maintained  for  addressing  purposes  (SYSl. USERS) 
Arguments  - 

userld(S)  - CHARACTER(8) 
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user 

user 

user 

user 

user 


name  - CHARACTER(20) 

group  - CHARACTER(IO) 

building  - CHARACTER(12) 

phone  no.  - CHARACTER{4) 

variable  Information  - CHARACTER( 16 ) 

- the  cost  centre  Is 
the  f I rst  3 bytes . 


I n 


(4)  LASTACC 

type  - PL/ I / subroutine 
called  from  - F0RCE2 

This  routine  obtains  the  date  of  last  access  of  a 
dataset/  in  Julian  form,  from  the  last  access  dataset 
( SYSl . DATASET . LASTACC ) . The  flag  byte  from  the  last 
access  dataset  is  also  returned.  if  there  is  no  entry 
for  the  dataset  the  flag  byte  is  set  to  X'80'. 
Arguments  - 

dataset  name(S)  - CHARACTER( 44 ) 
last  access  date  - PICTURE  9(5) 
flag  bits  - BIT(8) 

(5)  READACC 

type  - PL/I/  subroutine 
called  from  - GETVTOC 

This  routine  reads  the  last  access  dataset 
( SYSl . DATASET . LASTACC ) sequentially  and  at  each  call 
returns  the  next  record  from  It. 

Arguments  - 

dataset  name  - CHARACTER( 44 ) 
last  access  date  - PICTURE  9(5) 
flag  bits  - B I T( 8 ) 

end-of-flle  indicator  - CHARACTER(l) 

- this  is  non-blank  if  there  are  no  more 
records  in  the  dataset/  blank  otherwise. 


i 
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(6)  JULIAN 

entry  point  - UNJUL 

type  - PL/ I,  function  subprogram 

called  from  - GETVTOC, ARCHSET, D I RECUP, ARCH  I VE, EXP  I RY, 
P0STDS,F0P.CE2,LSTUSER,LSTADSN, 

LI STDSN,WARNING,LSTARCH 

This  function  returns  the  current  date  In  Julian 
form  (entry  point  JULIAN)  or  converts  a Julian  date 
to  the  form  dd/mm/yy  and  returns  this  character  string 
(entry  point  UNJUL). 

function  attributes  - PICTURE  9(5)  (entry  point  JULIAN) 

- CHARACTER(8)  (entry  point  UNJUL) 
Arguments  - Julian  date  to  be  converted  to  dd/mm/yy 

(entry  point  UNJUL  only)  (S)  - PICTURE  9(5) 

(7)  LNKMOVE 

type  - assembler,  subroutine 
called  from  - F0RCE2 

This  subroutine  performs  the  actual  data  transfer  to 
an  archive  tape  for  the  user  demand  archival  and  backup 
functions.  It  firstly  gains  control  of  the  archive  tape 
by  dynamically  allocating  the  dummy  dataset  at  the 
beginning  of  the  tape.  Then  the  utility  program  lEHMOVE 
is  invoked  to  perform  the  dataset  transfer.  A non-zero 
return  code  Indicates  that  the  request  failed. 

Arguments  - 

archive  tape  serial  number(S)  - PICTURE  9(6) 
return  code  - FIXED  BINARY(31,0) 

(8)  LNKRET 

type  - assembler,  subroutine 
called  from  - RETRVE 

This  subroutine  performs  the  actual  data  transfer  from 
tape  to  disk  for  the  retrieval  function.  Firstly  the 
dataset  to  be  retrieved  is  dynamically  allocated  on  the 
archive  tape  using  the  password  passed  as  a parameter. 
Then  the  utility  program  lEHMOVE  is  Invoked  to  perform 
the  dataset  transfer.  A non-zero  return  code  indicates 
that  the  request  failed. 

Arguments  - 

archive  tape  serial  number(S)  - PICTURE  9(6) 

password  of  dataset(S)  - CHARACTER(8) 

length  of  password(S)  - FIXED  BINARY(15,0) 

return  code  - FIXED  BINARY(51,0) 

dataset  to  be  retrieved(S)  - CHARACTER( 44 ) 

length  of  dataset  name(S)  - FIXED  BINARY(15,0) 

sequence  no.  on  archive  tape(S)  - FIXED  BINARY(15,0) 
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PREALLC 

type  - assembler/  subroutine 
called  from  - RETRVE 

This  routine  uses  dynamic  allocation  to  preallocate/ 
under  file-name  PREALL/  the  required  amount  of  space 
on  disk  In  preparation  for  retrieving  a sequential 
dataset.  This  is  necessary  because  I EHMOVE  cannot 
determine  how  much  space  the  tape  dataset  will  require  on 
disk  and  uses  a fairly  small  default  allocation.  This 
problem  does  not  exist  for  partitioned  datasets  since 
lEHMOVE  writes  control  Information  describing  the 
dataset's  space  allocation  when  It  is  unloaded  to  tape. 

A non-zero  return  code  indicates  that  the  space  could  not 
be  obtained. 

Arguments  - 

name  of  dataset(S)  - CHARACTER(44) 
length  of  dataset  name(S)  - FIXED  BINARY(15/0) 
disk  volume  serial  number(S)  - CHARACTER(6) 
primary  track  al locat lon(S)  - FIXED  BINARY(31,0) 
secondary  track  a 1 1 ocat I on ( S)  - FIXED  BINARY(31,0) 
return  code  - FIXED  BlNARY(31/0> 
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(10)  PROTADD 

entry  points  - PROTDEL , PROTREP, PROTLST, PROTAS, PROTDS 
type  - assembler,  subroutine 
called  from  - F0RCE2, RETRVE, SCRATCH 

This  module  performs  user  password  protection  functions 
for  datasets. 

Entry  point  PROTADD  adds  a control  password  with  read/ 
write  protection. 

Entry  point  PROTDEL  deletes  the  control  password  of  a 
dataset . 

Entry  point  PROTREP  replaces  the  control  password  of  a 
dataset  by  itself.  This  is  to  set  the  protect  bits  in 
the  dataset's  DSCB  on  disk  for  the  RETRIEVE/RELOAD 
funct ion . 

Entry  point  PROTEST  lists  the  information  for 
a dataset's  control  password.  This  is  to  verify  that 
the  password  is  correct. 

Entry  point  PROTAS  adds  a secondary  password  with  read/ 
write  protection. 

Entry  point  PROTDS  deletes  a secondary  password  for  a 
dataset . 

All  of  the  above  functions  use  the  PROTECT  macro 
instructionC ref .8)  and  pass  back  to  the  calling 
program  the  return  code  generated  by  this  macro. 
Arguments  - 

dataset  name(S)  - CHARACTER( 44 ) 

length  of  dataset  name(S)  - FIXED  BINARY(15,0) 

password(S)  - CHARACTER(8) 

return  code  - FIXED  BINARY(31,0) 

* volume  serial  no.  containing  dataset(S)  - CHARACTER(6) 
+ control  password(S)  - CHARACTER(8) 

(*  required  by  entry  points  PROTADD,  PROTDEL,  PROTREP, 
PROTAS  and  PROTDS  only). 

(+  required  by  entry  points  PROTAS,  PROTDS  only). 
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(11)  SECSPCE 

type  - assembler,  subroutine 
called  from  - GETVTOC, F0RCE2 

This  subroutine  reads  the  format  3 DSCB  of  a disk 
dataset  and  returns  the  number  of  tracks  occupied  by 
the  secondary  extents  described  by  this  control  block. 
(Uses  OBTAIN  macro  - see  reference  8). 

Arguments  - 

address  of  Format3  DSCB  (CCHHR)  (S)  - CHARACTER(5) 
disk  volume  serial  number(S)  - CHARACTER(6) 
tracks  - FIXED  BINARY(31,0) 

(12)  UPDATE 

entry  point  - UPJUL 

type  - PL/ I,  function  subprogram 

called  from  - ARCH  I VE, EXP  I RY, F0RCE2, WARN  I NG 

This  function  accepts  a date,  either  In  the  form 

dd/mm/yy  (entry  point  UPDATE)  or  Julian  (entry  UPJUL) 

and  an  Integer  number.  It  adds  this  number  of  days  to 

the  date  and  returns  the  resultant  date  In  the 

appropriate  form. 

function  attributes  - entry  point  UPDATE  - CHARACTER(8) 

- entry  point  UPJUL  - PICTURE  9(5) 

Arguments  - 

first  date(S)  - CHARACTER(8)  for  UPDATE 
- PICTURE  9(5)  for  UPJUL 
no.  days  to  add  (S)  - FIXED  BINARY  (31,0) 
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(13)  USERSMF 

type  - assembler,  subroutine 

called  from  - ARCHSET, SHI FTI D,ARCHUP,EXPI RY, SCRATCH, 
RETRVE , LSTARCH, UPCAT, CHNGVOL, DUMPSMF 
This  subroutine  Is  called  whenever  a change  Is  made  to 
the  archive  catalogue.  It  writes  a user  record  to  the 
SMF  dataset  containing  the  new  or  updated  archive 
catalogue  record.  The  type  of  alteration  Is  also 
Indicated.  The  routine  is  also  called  by  DUMPSMF  to  just 
copy  the  contents  of  the  archive  catalogue  to  the  SMF 
dataset.  Any  load  module  that  includes  this  subroutine 
must  reside  In  an  authorized  program  library  and  Itself 
be  authorized. 

Format  of  SMF  records 


Offset  Size 


Descr I pt ion 


0 1 
1 1 


2 4 

6 4 


10  4 

14  8 

22  80 


system  I nd icator ( =X ' 02 ' - VS2) 
record  type  (binary)  - 

128  : addition  by  migration  or 

demand  arch  I val . 

129  : reload  - this  record  does 

not  indicate  a change  to  the 
archive  catalogue.  It  is 
included  only  for  gathering 
usage  statistics. 

130  : deletion  (by  scratching, 

replacement  or  expiry  date 
lapsing) . 

131  : alteration 

132  : addition  by  backup 

133  : retrieval 

134  ; generated  by  DUMPSMF  for 

usage  statistics  and 
catalogue  recovery.  It  does 
not  indicate  a change  to  the 
catalogue . 

time  of  day  record  was  written.  In 
hundredths  of  a second  (binary) 
date  record  was  written  In  Julian 
form  (packed  decimal) 
machine  identifier  (=C*W168') 
name  of  job 

contents  of  archive  catalogue 
record.  For  an  alteration  this  is 
the  new  contents. 


Arguments  - 

archive  catalogue  record(S)  - CHARACTER( 80 ) 
record  type(S)  - FIXED  DECIMAL(3,0) 
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(14)  DSNCAT 

type  “ assembler/  subroutine 
called  from  - F0RCE2 

This  subroutine  obtains  the  volume  and  device  type 
Information  for  a dataset  directly  from  the  operating 
system  catalogue.  (Uses  LOCATE  macro  - see  reference  8). 
If  the  dataset  Is  not  found  in  the  catalogue  then  the 
volume  and  device  type  fields  are  both  returned  as  blank. 
Arguments  - 

dataset  name(S)  - CHARACTER( 44 ) 

volume  dataset  resides  on  - CHARACTER(6) 

device  type  code  of  volume  - CHARACTER(4) 

(15)  DELDSN 

type  - assembler,  subroutine 
called  from  - F0RCE2/ RETRVE 

This  subroutine  accepts  a dataset  name  and  disk  volume 
serial  no.  and  attempts  to  uncatalogue  the  dataset  and 
delete  It  from  the  volume. 

Arguments  - 

dataset  name(S)  - CHARACTER( 44 ) 
volume  serial  number(S)  - CHARACTER(6) 

(16)  DSNVTOC 

type  - assembler,  subroutine 
called  from  - F0RCE2 

This  routine  obtains  a dataset's  format  1 DSCB  directly 
from  a disk  volume  VTOC.  (Uses  OBTAIN  macro  - see 
reference  8). 

Arguments  - 

dataset  name(S)  - CHARACTER( 44 ) 

volume  dataset  resides  on(S)  - CHARACTER(6) 

work  area  - CHARACTER( 140 ) 

- the  first  96  bytes  will  contain  the  data  portion 
of  the  format  1 DSCB,  the  next  5 bytes  the 
absolute  track  address  (CCiiHR)  of  the  DSCB. 

return  code  - FIXED  BINARY(31,0) 

- non-zero  indicates  that  the  request  failed. 
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(17)  DSCBl 

type  - assembler,  subroutine 
called  from  - VTOCDSN 

This  subroutine  reads  all  format  1 DSCB's  from  a 
nominated  disk  volume,  returning  one  at  each  call.  Once 
one  VTOC  has  been  exhausted  the  subroutine  can  be  called 
with  a different  volume,  but  not  before.  Each  volume  Is 
dynamically  allocated  and  de-al located  with  a DDNAME  of 
FULLVTOC.  At  each  call  the  free  space  on  the  current 
volume  is  also  returned. 

Arguments  - 

disk  volume  serial  no.(S)  - CHARACTER(6) 
format  1 DSCB  - CHARACTER( 140 ) 
free  cylinders  on  volume  - FIXED  BINARY(31,0) 
free  tracks  on  volume  - FIXED  BINARY(31,0) 
return  code  - FIXED  BINARY(31,0) 

- contains  0 if  the  call  was  successful. 
Is  positive  If  It  was  unsuccessful  and 
is  -1  If  there  are  no  more  format  1 
DSCB's  on  this  volume.  (This  Indicates 
that  another  volume  can  now  be 
spec  I f I ed ) . 

(18)  ADDRESS 

type  - PL/ I,  subroutine 
called  from  - LSTWARN, LSTARCH 
calls  - GETUSER 

This  routine  formats  and  writes  a page  containing  the 
name  and  address  of  a user  to  a report  file  (the  name 
of  which  Is  passed  as  a parameter). 

Arguments  - 

userld(S)  - CHARACTER(8) 

output  file  name(S)  - FILE  OUTPUT  STREAM  PRINT 
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(19)  ARCHSET 

type  - PL/ I , subroutine 
called  from  - ARCH  I VE/ F0RCE2 
, calls  - JULIAN, USERSMF 

I This  routine  formats  and  writes  lEHMOVE  control 

statements  to  copy  a dataset  to  an  archive  tape  and/or 
. lEHPROGM  control  statements  to  uncatalogue  and  delete 

the  dataset  from  disk.  In  addition,  depending  on  input 
parameters,  a record  may  be  added  to  the  archive 
catalogue  or  replace  an  existing  one,  in  which  case  the 
: appropriate  SMF  records  are  also  written. 

■ Arguments  - 

’ archive  catalogue  record(S)  - CHARACTER(80) 

replacement  Indicator  - CHAR^CTER(4)  - This  will  be  set 
to  *REPL'  if  the  record  replaces  one  of  the 
same  name  In  the  archive  catalogue,  blanks 
otherwl se 

function  Indicator(S)  - FIXED  DECIMAL(1,0) 

- if  0 then  update  the  archive  catalogue 
and  generate  lEHMOVECMOVE  DSNAME)  control 
statement 

- if  1 then  only  update  the  archive 
catalogue 

- if  2 then  generate  lEHMOVECCOPY  DSNAME) 

1 and  lEHPROGM  control  statements  only 

- if  4 then  generate  lEHPROGM  control 
statements  only 

type  of  data  transfer(S)  - CHARACTER(4) 

- if  'ARCH*  then  the  operation  is  a dataset 
migration  or  demand  archival 

I - if  'BACK'  then  the  operation  is  a dataset 

backup. 
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(20)  GETVTOC 

type  - PL/ I,  subroutine 

called  from  - ARCH  I VE , L I STDSN, WARN  I NG, USERSPC 
calls  - SECSPCE, JULIAN, LASTACC 

GETVTOC  uses  VTOC  Information  generated  by  program 
VTOCOSN  (see  Appendix  11.2.1  (2))  to  extract  dataset 
properties.  At  each  call  detailed  information  about  the 
next  dataset  in  the  VTOC  input  Is  returned.  The  most 
Important  Information  includes  the  dataset  name,  the 
volume  it  resides  on,  its  space  requirements,  the  date 
it  was  last  accessed  and  whether  it  was  flagged  for 
migration  (both  obtained  from  the  last  access  dataset  - 
SYSl. DATASET. LASTACC) . If  there  is  no  entry  in 
SYSl . DATASET . LASTACC  then  the  current  date  is  returned 
as  the  last  access  date. 

Arguments  - 

dataset  name  - CHARACTER( 44 ) 
disk  volume  serial  number  - CHARACTER(6) 
tracks  dataset  occupies  - PICTURE  9(5) 
dataset  creation  date  (Julian)  - PICTURE  9(5) 
dataset  retention  date  (Julian)  - PICTURE  9(5) 
date  of  last  access  (Julian)  - PICTURE  9(5) 
dataset  organization  code  - CHARACTER(2) 
number  of  volumes  so  far  encountered  - 
FIXED  BINARY(31,0) 

all  volume  serial  numbers  so  far  encountered  - 
(10)  CHARACTER(6) 

free  tracks  on  all  volumes  encountered  - 
(10)  FIXED  DECIKAL(4,0) 
free  cylinders  on  all  volumes  encountered  - 
(10)  FIXED  DECIMAL(4,0) 
dataset  flag  bits  - 3IT(16) 

- bits  0-7  are  the  flag  bits  from 
SYSl. DATASET. LASTACC. 

- bit  8 is  on  if  the  dataset  is 


empty 

- bits  9-10  are  the  dataset 
protection  bits  from  the  field 
DSIDSIND  of  the  format  1 DSCB. 

- bits  11-15  are  unused  (zero) 
end-of-file  indicator  - this  is  non-blank  if  there  are 

no  more  VTOC  entries,  blank 
otherwi se . 
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11.2  Main  proRrauis 


The  main  program  modules  are  described  next,  grouped  by 
function.  The  file  names  used  by  each  program  are  also 
1 i sted . 

In  addition  some  programs  dynamically  Invoke  others  and 
when  these  are  executed  the  Hies  used  by  the  Invoked 
programs  must  also  be  supplied.  The  programs  Involved  are 
1 is  ted  bt:low. 


Invoked  program  Calling  programs 


File  N^mes 


POSTDS 
I EHMOVE 


SORT 


ARCHGDG 

RETRVE(vIa  LNKRET), 
F0RCE2(vla  LNKMOVE) 


VTOCDSN,SMF1415, 

USERSPC,TAPEMAP, 

TAPMRGE 


SYSIN,LASTACCa- 
SYSPRINT(see  11.2.4(6)) 
SYSUT1,SYSPRINT, 
SYSIN,USERn,DDn. 

The  USERn  ddnames  each 
describe  one  of  the 
user  packs  SAOOOn,  and 
the  DDn  ddnames  one  of 
the  system  packs.  There 
must  be  one  of  these 
statements  for  each 
volume  that  is  a 
candidate  for  demand 
archival,  backup  or 
ret  r i eval . 

The  I EHMOVE  DD  state- 
ments must  be  placed 
before  those  of  the 
call i ng  program. 

SORTLI B,SYSOUT, 
SORTWKOn. 


All  main  programs  used  in  catalogued  procedures  (see  Appendix 
III),  as  well  as  those  that  call  Subroutine  USERSMF  (and  must 
therefore  be  authorized),  are  fully  link-edited  as  executable 
load  modules  in  the  dataset  SYSl.WREL I NK.  The  module  names 
all  begin  with  the  4 characters  ARCH.  Care  must  be  taken  to 
ensure  that  whenever  a load  module  is  replaced  in 
SYSl. ARCHIVE. LOAD  all  modules  in  SYSl.WRELINK  that  use  it  are 
relinked  to  include  the  new  version.  The  module  names  of 
those  main  programs  that  are  in  SYSl.WRELINK  are  indicated  in 
this  Appendix.  The  remaining  programs  are  run  only  when 
required  and  must  be  relinked  from  SYSl . ARCH  I VE . LOAD  each 
time. 
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M.2.1  Stage  1 modules 

(1)  SMF1415 

type  - PL/ I 

load  module  - ARCHSMFD 

This  program  extracts  access  records  (types  14,  15  and 
140)  from  the  SMF  data,  sorts  them  by  date  within  dataset 
name/  and  writes  them  to  an  output  dataset.  Access 
records  for  SYSl  datasets  are  ignored.  A second  input 
dataset  may  specify  a time  stamp  before  which  SMF  records 
are  disregarded.  The  program  always  updates  this  time 
stamp  in  the  dataset  with  that  of  the  last  SMF  record 
read  from  the  input. 

Input  formats  - 

(a)  File  SMF I N 

All  SMF  records  generated  by  the  system  (including 
type  140  - see  Section  4.1).  For  the  format  of 
other  SMF  records  see  reference  6. 

(b)  File  LASTSMF  - member  LASTSMF  of 

SYSl . ARCH  I VE . PARML I B . This  single  80-byte  record 
dataset  contains  the  time  stamp  of  the  last  SMF 
record  processed  by  this  program.  Records 
generated  before  this  time  will  not  be  considered. 
The  program  also  updates  this  time  stamp  when  all 
SMF  records  have  been  processed. 

Offset  Size  Description 

0 4 time  of  day  record  was  written, 

in  hundredths  of  a second 
(binary) 

4 4 date  record  was  written,  in 

Julian  form  (packed  decimal) 

8 72  unused 

Output  format  - 
(a)  File  SORTOUT 

Types  14,  15  and  140  SMF  records. 
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(2)  VTOCDSN 

type  - PL/ I 

calls  - DSCBl 

load  module  - ARCHVDSN 

This  program  reads  the  VTOCs  of  one  or  more  disk  volumes 
(via  DSCBl)  and  produces  output  records  containing 
dataset  name,  the  format  1 DSCB,  the  volume  serial  no. 
and  the  volume  free  space  information.  These  records  are 
sorted  on  dataset  name.  Note  that  this  program  is  also 
used  in  Stage  2 and  to  supply  input  to  all  other  programs 
that  call  GETVTOC. 


Input  format  - 

(a)  File  DISKS  - member  DISKS  of  SYSl . ARCH  I VE . PARML I B 

This  file  indicates  the  disk  volumes  whose  VTOCs  are 
to  be  read.  There  Is  one  80-byte  record  for  each 
volume,  with  the  serial  number  beginning  in  the  first 
byte . 


Output  format  - 
(a)  File  SORTOUT 

These  are  154-byte  records  containing  dataset 
Information  in  sorted  order. 

The  format  is  as  follows  - 


Offset  Size 


Desc  r i pt I on 


0 44 

44  96 


140  6 

146  4 

150  4 


dataset  name 
the  data  portion  of  the 
dataset's  format  1 DSCB 
the  disk  volume  serial  no. 
free  cylinders  on  the  volume 
- numeric,  binary 
free  tracks  on  the  volume  - 
numeric,  binary 
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(3)  DIRECUP 

type  - PL/ I 

calls  - JULIAN 

load  module  - ARCHUPDD 

This  module  uses  the  sorted  SMF  access  data  created  by 
SMF1415,  the  sorted  dataset  records  produced  by  VTOCDSN 
and  a copy  of  the  current  last  access  dataset 
(SYSl. DATASET. LASTACC. BACKUP)  to  produce  the  new  last 
access  dataset.  Accesses  by  exempt  Jobs  (also  specified 
via  an  input  dataset)  are  ignored.  In  addition  a bit  is 
set  in  the  flag  byte  for  datasets  that  have  type  15 
records  In  the  SMF  input,  indicating  update  activity. 

This  program  can  also  perform  the  initial  load  of  the 
last  access  dataset. 

Input  formats  - 

(a)  PARM  field 

[ load] 

If  LOAD  is  specified  then  this  is  the  initial  load 
of  SYSl. DATASET. LASTACC. 

(b)  File  SMF 

SMF  record  types  14,  15  and  140,  as  output  by 
program  SMF1415  in  file  SORTOUT. 

(c)  File  VTOC 

A file  of  154-byte  records  containing  dataset  names 
and  VTOC  information  as  output  in  file  SORTOUT  by 
prograru  vtOCDSN. 

(d)  File  DIRECIN  - a copy  of  the  last  access  dataset. 

( SYSl . DATASET . LASTACC . BACKUP) 

This  file  Is  not  used  if  this  is  the  initial  load  of 
the  last  access  dataset. 

(e)  File  XMPTJOB 

This  dataset  contains  the  names  of  the  exempt  jobs. 
There  is  one  80-byte  record  for  each,  with  the  Job- 
name  in  bytes  1 to  8 . These  are  contained  in  member 
XMPTJOB  of  SYSl.ARCHIVE.PARMLI B. 

Output  format  - 

(a)  File  DIREC  - the  new  last  access  dataset 


(SYSl. DATASET. LASTACC) 

This  dataset  should  be  empty  initially. 


37 


WRE-TR-1870(A) 


(4)  WARNING 

type  - PL/ I 

calls  - GETVTOC, ELAPSED, UPDATE, JULIAN, ENQDEQ 
load  module  - ARCHWARN 

This  Is  the  major  program  In  Stage  1 of  the  automatic 
migration  software.  Its  purpose  Is  to  perform  the 
dataset  selection.  The  parameter  Input  Is  the  maximum 
number  of  weeks  a dataset  can  remain  on  disk  without 
being  accessed,  the  percentage  free  space  required  and 
the  number  of  days  until  Stage  2 Is  to  be  run. 

Additional  variable  Input,  via  datasets,  are  the  names 
of  selected  datasets  and  userids  whose  datasets  must 
remain  on  disk  and  a series  of  numbers  representing 
track-day  ranges. 

The  program  reads  the  VTOCs  of  the  participating  disk 
volumes  (via  GETVTOC).  Datasets  that  are  not  sequential 
or  partitioned  or  that  are  exempt  from  migration  are 
rejected.  Any  datasets  that  have  been  flagged  for 
migration  or  have  not  been  accessed  for  the  period 
specified  by  the  input  parameter  are  selected. 

For  all  the  remaining  datasets  a track-day  value  is 
calculated  - currently  by  multiplying  the  size,  in 
tracks,  by  the  number  of  days  since  the  last  access, 
although  this  can  easily  be  altered.  Details  of  the 
datasets,  including  the  track-day  values,  are  written  to 
a temporary  file.  During  this  process  the  space  occupied 
by  datasets  is  accumulated  by  the  track-day  ranges 
spec i f led . 

When  all  datasets  have  been  processed  the  total  free 
space  currently  on  disk,  and  hence  the  amount  that  needs 
to  be  released,  can  be  determined.  If  sufficient  space 
has  already  been  made  available  by  the  first  selection 
criterion  then  no  further  selection  is  required. 

However,  if  this  is  not  the  case  then  all  datasets  whose 
track-day  figure  falls  in  the  highest  valued  range  are 
selected.  Successively  lower  valued  ranges  are  selected 
until  no  further  space  Is  required.  The  dataset  names 
are  determined  by  reading  back  the  temporary  file 
produced  during  the  VTOC  scan. 
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Finally  a file  (SYSl . ARCH  I VE . WARNL I ST)  containing  details 
of  all  selected  datasets  Is  written.  The  first  record 
In  this  file  Is  always  the  date  that  stage  2 will  be  run. 

Input  formats  - 

(a)  FARM  field 

max.  weeks,  free  space,  days 

If  max.  weeks  Is  zero  then  selection  Is  performed 
only  by  track-day  values. 

(b)  File  VTOC 

A file  of  154-byte  records  containing  dataset  names 
and  VTOC  Information  as  output  in  file  SORTOUT  by 
program  VTOCDSN. 

(c)  File  DIREC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

(d)  File  SYSIN 

There  are  two  forms  of  Input  via  this  file.  The 
first  type  is  a list  of  userids  and/or  datasets  that 
are  exempt  from  migration.  A maximum  of  50 
exemptions  may  be  specified.  There  Is  one  80-byte 
record  for  each,  with  the  userid  or  dataset  name 
starting  In  the  first  byte.  These  are  contained  In 
member  XMPTSTEM  of  SYSl . ARCH  1 VE . PARML I B . The  second 
type  of  Input  Is  the  list  of  track-day  ranges,  one 
In  each  80-byte  record,  with  format  X,9(4).  A 
maximum  of  50  values  may  be  specified  and  they  must 
be  In  ascending  numeric  order,  but  may  be  Inter- 
spersed with  the  exempt  userid  and  dataset  records. 
They  are  contained  in  member  TRKDAYS  of 
SYSl .ARCH I VE . PARML I B.  If  no  track-day  records  are 
supplied  then  selection  is  performed  only  by  the 
max.  week  criterion  specified  in  the  PARM  field  and 
this  may  not  be  sufficient  to  meet  the  stated  free 
space  objective. 

Output  formats  - 

(a)  File  NEWLIST  - SYSl . ARCH  I VE .WARNL I ST 

(b)  File  SYSPRINT 

Various  statistics  about  the  selection  criteria  are 
printed  on  this  file. 
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(5)  LSTWARN 

type  - PL/ I 

calls  - ADDRESS 

load  module  - ARCHLSTW 

This  program  prints  a report  for  both  users  and  dataset 
manager  Indicating  the  names  of  the  datasets  that 
have  been  selected  for  migration  and  the  date  that  It 
will  occur. 

Input  formats  - 

(a)  File  NEWLIST  - SYSl . ARCH  I VE . WARNL I ST 

(b)  File  USERS  - the  users'  details  file  (SYSl. USERS) 

Output  formats  - 

(a)  File  SYSREPT 

The  dataset  manager  report  Is  written  to  this  file. 

(b)  File  USEREPT 

The  user  report  Is  written  to  this  file. 
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11.2.2  Stage  2 modules 

(1)  ARCHIVE 

type  - PL/ I 

calls  - GETVTOC, ARCHSET, UPDATE, JULIAN 
load  module  - ARCHIV  (must  be  authorized) 

ARCHIVE  is  the  main  program  In  Stage  2 of  the  automatic 
migration  software.  The  parameters  Input  are  the  archive 
tape  serial  number,  the  first  sequence  number  to  be 
written  to  the  tape  and  the  default  retention  period  to 
be  applied  to  the  datasets  about  to  be  migrated. 

The  program  uses  VTOC  Information  from  the 
participating  disk  volumes  (via  GETVTOC)  to  determine  the 
full  details  of  those  datasets  selected  for  migration  that 
still  remain  on  disk  as  well  as  any  that  users  have 
flagged  for  migration  since  the  running  of  Stage  1.  Via 
subroutine  ARCHSET  the  program  prepares  I EHMOVE  control 
statements  to  move  these  datasets  to  successive  sequence 
numbers  on  tape.  A record  is  also  added  to  the  archive 
catalogue  for  each  dataset. 

The  exception  to  this  occurs  for  datasets  that  are 
empty.  They  are  simply  deleted  (via  lEHPROGM)  and  not 
migrated. 

Next  a file  containing  the  names  of  the  newly-migrated 
datasets  is  created  ( SYSl . ARCH  I VE . ARCHL I ST ) . Finally 
another  file  indicating  the  archive  tape  serial  number 
and  the  number  of  datasets  on  it  is  written  to  file 
ARCHTAP  (for  use  by  the  user  demand  archival  and  backup 
funct ions) . 

The  program  also  turns  off  all  bits  of  the  flag 
byte  in  the  SYSl . DATASET . LASTACC  entries  for  migrated 
datasets,  indicating  that  the  request  has  now  been 
fulfil  led. 
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Input  formats  - 

(a)  FARM  field 

tape  serial,  sequence  no.,  retention  period  (months) 

(b)  File  VTOC 

A file  of  154-byte  records  containing  dataset  names 
and  VTOC  Information  as  output  In  file  SORTOUT  by 
program  VTOCOSN. 

(c)  File  DIREC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

(d)  File  OLDLIST  - SYSl.ARCHIVE.WARNLIST 

(e)  File  ARCHCAT  - the  archive  catalogue 
(SYSl.ARCHIVE.CATLG) 

Output  formats  - 

(a)  File  NEWARCH  - SYSl . ARCH  I VE .ARCHL I ST 


(b) 


(c) 

(d) 

(e) 


File  ARCHTAP  - member  ARCHTAP  of 

SYSl. ARCHIVE. PARMLIB.  The  format  is  as  follows 


Offset  Size 


0 

6 

12 


6 

6 

68 


Description 

current  archive  tape  serial  number 
next  sequence  number  to  be  used 
(numeric,  display  format) 
unused 


File  lEHMOV 

I EHMOVE  control  statements  (80-byte  records). 

File  lEHPROG 

lEHPROGM  control  statements  (80-byte  records). 

File  SYSPRINT 

Statistics  for  the  program  run  are  written  to  this 
file. 
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(2)  LSTARCH 

type  - PL/ I 

calls  - ADDRESS, USERSMF, ELAPSED, JULIAN 
load  module  - ARCHLSTA  (must  be  authorized) 

This  program  produces  a report  for  both  the  dataset 
manager  and  users  reflecting  the  current  state  of 
the  archives.  The  manager's  report  simply  lists  the 
complete  archive  catalogue  and  is  optional,  being 
selected  or  not  by  specifying  Y or  N respectively  as 
the  first  of  two  parameters. 

The  second  parameter  specifies  PART  or  FULL.  For  the 
latter  every  user  receives  a list  of  the  names  and 
expiry  dates  of  ail  his  datasets  currently  in  the 
archives.  With  PART  only  those  users  who  have  had 
datasets  added  to  the  archives  during  this  run,  or  have 
datasets  with  less  than  one  month  until  expiry,  or 
have  had  datasets  expire  during  the  past  week  receive  a 
report.  Ail  expired  datasets  are  deleted  from  the 
archive  catalogue  by  this  program. 


Input  formats  - 
(a)  PARM  field 


(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

(c)  File  NEWARCH  - SYSl.ARCHI VE.ARCHLI ST 

(d)  File  USERS  - the  users'  details  file  (SYSl. USERS) 

Output  formats  - 

(a)  File  SYSREPT 

The  dataset  manager's  report  indicating  the  datasets 
migrated  this  run. 

(b)  File  SYSLIST 

The  optional  dataset  manager's  report  listing  the 
ent i re  catalogue . 

(c)  File  USEREPT 

The  users'  archive  status  reports. 
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.3  Load  balancing  modules 

(1)  USERSPC 
type  - PL/ I 
calls  - GETVTOC 

load  module  * ARCHUSPC 

This  program  uses  VTOC  information  to  produce  a report 
indicating  the  space  owned  by  each  user  on  each  disk 
volume.  This  report  Is  used  to  select  users  for  transfer 
from  heavily  loaded  volumes. 

Input  formats  - 

(a)  File  VTOC 

A file  of  154-byte  records  containing  dataset  names 
and  VTOC  Information  as  output  In  file  SORTOUT  by 
program  VTOCDSN. 

(b)  File  DIREC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

Output  format  - 

(a)  File  SYSPRINT  - the  space  report. 

(2)  SHI  FT  ID 
type  - PL/ I 

calls  - USERSMF.ENQDEQ 

load  module  - ARCHSHFT  (must  be  authorized) 

This  program  generates  utility  control  statements  to 
transfer  users  from  one  volume  and/or  system  catalogue  to 
another.  There  are  four  sources  of  input.  The  first  Is 
a constant  dataset  listing  each  user  disk  volume  serial 
number,  the  serial  number  of  its  paired  volume,  the 
symbolic  unit  name  assigned  to  the  pair  and  the  name  of 
the  associated  user  catalogue. 

The  second  input  is  a list  of  userids  to  be  shifted, 
the  volutTies  they  are  currently  assigned  to,  the  volumes 
they  are  to  be  shifted  to  and  optionally  an  exempt  volume 
for  each. 

The  third  input  dataset  is  a copy  of  each  catalogue  in 
the  system,  as  generated  by  program  CATCOPY  (ref. 12). 

The  final  Input  source  Is  a parameter  specifying 
whether  this  is  a test  run  (NOSYS)  or  a production  run 
(SYS),  in  which  case  the  disk  volume  entries  for  each 
user's  archive  catalogue  records  are  updated  to  reflect 
the  new  preferred  volumes.  Subsequent  retrievals  will 
therefore  be  directed  to  these  volumes. 
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The  program  reads  the  catalogue  (via  CATLGRD)  and 
generates  lEHMOVE  control  statements  to  copy  NONVSAM 
datasets  belonging  to  the  selected  users  to  the  indicated 
volumes  (except  those  already  on  the  volumes  and  those  on 
the  exempt  volumes).  iDCAMS  control  statements  to  delete 
the  original  datasets  and  perform  catalogue  maintenance 
are  also  generated  as  are  statements  for  changing  the 
UADS  entries  of  users  whose  disk  unit  name  has  changed. 
VSAM  catalogue  entries  (including  Generation  Data 
Groups)  are  listed  for  further  Investigation. 

Finally,  for  production  runs  only  (SYS),  the  disk 
volume  field  of  each  archive  catalogue  record  of  each 
selected  user  is  updated. 

Input  formats  - 
(a)  FARM  field 


(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

(c)  File  CATLG 

A sequential  dataset  containing  a copy  of  one  or 
more  of  the  system  catalogues.  This  Is  read  by  sub- 
routine CATLGRD,  described  in  reference  12. 

(d)  File  SYSIN 

There  are  two  types  of  Information  in  this  file 
of  80-byte  records.  They  may  be  input  in  any 
order.  The  first  type  of  record  is  detailed  below. 
There  is  one  for  each  user  volume  in  the  system. 


Offset 

Size 

Descr i pt i on 

0 

2 

unused 

2 

6 

The  volume  serial  number 

8 

2 

unused 

10 

16 

The  serial  number  of  the  paired 
volume 

16 

2 

unused 

18 

5 

The  symbolic  unit  name  of  the 
pair  of  vol umes 

23 

3 

unused 

26 

44 

The  user  catalogue  associated 
with  the  volume 

70 

10 

unused . 
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The  second  record  type  Is  described  below.  There  Is  one  j 

record  for  each  userid  to  be  shifted.  | 

Offset  Size  Description  | 

the  userid 

unused  I 

the  volume  currently  assigned 
to  the  user 
unused 

the  volume  the  user  Is  to  be 
shifted  to  , 

unused 

an  exempt  volume  from  which 
datasets  will  not  be  shifted 
(optional) 

unused.  i 

i 

Output  formats  - ! 

(a)  File  lEHMOVE 

lEHMOVE  control  statements  (80-byte  records)  to 
copy  the  datasets. 

(b)  File  IDCAMSl 

IDCAMS  control  statements  to  delete  and  uncatalogue  j 

the  original  datasets  (80-byte  records) 

(c)  File  IDCAMS 

IDCAMS  control  statements  to  recatalogue  the  NONVSAM 
datasets  (80-byte  records) 

(d)  File  UADS 

The  TSO  ACCOUNT  commands  required  to  change  the 
UADS  unit  field  for  users  shifted  to  a different 
volume  pair. 

(e)  File  SYSPRINT 

Messages  are  printed  for  datasets  and  catalogue 
entries  that  cannot  easily  be  manipulated 
automatically  (e.g.  VSAM). 
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(3)  CHNGVOL 

type  - PL/ I 

calls  - USERSMF.ENQDEQ 

load  module  - ARCHCVOL  (must  be  authorized) 

This  program  replaces  the  disk  volume  field  In  selected 
archive  catalogue  records  with  a volume  serial  number 
specified  In  the  FARM  field.  The  dataset  names  may  be 
Input  via  file  SYSIN  or  through  the  FARM  field  (one  name 
or  name  stem  only).  When  the  latter  form  is  used  the 
catalogue  records  of  all  datasets  with  names  beginning 
with  the  character  string  supplied  (which  may  be  a 
userid/  for  example)  are  updated.  The  dataset  manager 
can  use  the  program  to  force  retrievals  for  the  selected 
datasets  or  user  to  be  directed  to  a specific  volume. 
Although  this  offers  no  immediate  benefits  it  can  be 
considered  a form  of  delayed  load  balancing. 

Input  formats  - 

(a)  FARM  field 

disk  volume,  /SELECT  \ 

\dataset  name  stem/ 

SELECT  indicates  that  the  dataset  names  are 
supplied  In  file  SYSIN. 

(b)  File  SYSIN 

If  SELECT  was  specified  as  the  second  parameter 
In  the  FARM  field  this  dataset  contains  80  byte 
records  with  one  dataset  name  in  each,  beginning 
in  the  first  byte. 

(c)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 
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11.2.4  User  facility  modules 

(1)  RETRVE 

type  - PL/ I 

calls  - PREALLC,ENO.nEQ,DELDSN,USERSMF,LNKRET,PROTADD 
load  module  - ARCHRET  (must  be  authorized) 

This  program  performs  dataset  retrievals  and  reloads. 

The  Input  parameters  are  the  dataset  name,  the  type  of 
operation  (RET  for  RETRIEVE  or  REL  for  RELOAD)  and 
optionally  a password.  If  the  password  supplied  Is  not 
correct  (I.e.  Is  not  the  control  password  of  the  dataset) 
or  If  the  dataset  Is  protected  and  no  password  was 
provided,  then  the  retrieve  will  fall. 

If  the  dataset  Is  not  protected  (I.e.  does  not  have  an 
entry  In  the  PASSWORD  dataset)  then  a dummy  password  Is 
created  for  It  (entry  PROTADD).  This  must  be  done 
before  the  dataset  can  be  accessed. 

Any  catalogue  entry  or  dataset  of  the  same  name  already 
existing  on  the  receiving  volume  are  first  deleted.  Next, 
If  the  dataset  to  be  retrieved  is  sequential  then  the 
required  space  is  preallocated  on  the  receiving  disk 
volume.  lEHMOVE  control  statements  to  copy  the  dataset 
to  disk  and  lEHPROGM  control  statements  to  catalogue  it 
are  created.  Next  the  dataset  is  actually  copied  to  disk 
(entry  LNKRET)  and,  if  a retrieval  rather  than  a reload, 
the  record  is  removed  from  the  archive  catalogue. 

Finally,  If  the  dataset  was  not  protected  then  the  dummy 
password  Is  deleted  (entry  PROTDEL),  otherwise  the 
protect  bits  are  set  In  the  dataset  control  block  on  disk 
(entry  PROTREP) . 
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Input  formats  - 

(a)  FARM  field 
dsname,/RET' 
\REL 


/passworch 


(b) 


indicates  that  no  password  was  supplied. 

File  ARCHCAT  - the  archive  catalogue 
(SYSl.ARCHI VE.CATLG) 


Output  formats  - 

(a)  File  lEHMOVE 

I EHMOVE  control  statements  to  copy  the  dataset  to 
disk  (via  entry  LNKPET)  are  generated  in  this  file. 

(b)  File  lEHPROG 

lEHPROG  control  statements  to  catalogue  the  dataset 
on  disk  are  generated  in  this  file. 

(c)  File  SYSPRIN 

A message  Indicating  the  result  of  the  request  is 
generated  here. 
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SCRATCH 
type  - PL/ I 

calls  - USERSMF,ENQDEa,PROTADD 

load  module  - ARCHSCR  (must  be  authorized) 

SCRATCH  Is  used  to  delete  datasets  from  the  archives. 

The  parameters  Input  are  the  dataset  names  and  optionally 


me  parameters  input  are  tne  dataset  names  and  option 
a password  for  each.  If  a password  Is  supplied  It  Is 
checked  for  correctness  (entry  PROTEST).  If  the  password 
Is  Incorrect/  or  If  the  dataset  Is  protected  and  no  pass- 
word was  supplied  then  that  dataset  will  not  be  deleted. 
The  dataset  names  and  passwords  may  be  Input  via  file 
SYSIN  and/or  through  the  PARM  field  (one  name  and 
password  only). 


Input  formats  - 
(a)  PARM  field 

(8  sname\/ /password'^ 
\ NONE  / \ & i 


NONE  Indicates  that  no  dataset  name  is  specified 
In  the  PARM  field. 

Indicates  that  no  password  was  supplied. 


File  SYSIN 

One  80-byte  record  per  dataset,  with  the  name 
beginning  In  byte  1 and  optionally  a password 
separated  from  the  dataset  name  by  one  blank. 

Note  that  a special  Input,  of  the  form  userid(ALL), 


or  user  Id . (ALL ) , may  be  specified  as 
(either  In  the  PARM  field  or  In  file 
indicate  that  all  datasets  belonging 
specified  userid  are  to  be  scratched 
archives.  Of  course  only  those  that 


arcnives.  ur  course  or 
or  those  that  have  the 
supplied  are  deleted. 


a dataset  name 
SYSIN)  to 
to  the 
from  the 
are  unprotected 


password  as 


(c)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) . 

Output  format  - 
(a)  File  SYSPRIN 

Messages  indicating  the  result  of  the  request 
printed  for  each  dataset. 
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(3)  EXPIRY 

type  - PL/ I 

calls  - ELAPSED, UPDATE, JULIAN, USERSMF,ENQDEQ 
load  module  - ARCHEXP  (must  be  authorized) 

This  program  increases  the  expiry  date  of  datasets  In 
the  archives.  The  input  parameters  are  the  dataset  names 
and  the  number  of  months  the  date  of  each  Is  to  be  In- 
creased. No  passwords  are  required.  However  the  expiry 
date  may  not  be  more  than  1 year  In  advance  at  any  time. 
The  dataset  names  may  be  Input  via  file  SYSIN  and/or 
through  the  PARM  field  (one  name  only). 

Input  formats  - 

(a)  PARM  field 
/dsnam^  months 
\ NONE  / 

NONE  Indicates  that  no  dataset  name  is  specified 
in  the  PARM  field. 

(b)  File  SYSIN 

One  80-byte  record  per  dataset,  with  the  name 
starting  In  byte  1.  Note  that  a special  input  of 
the  form  userid(ALL),  or  user  id . (ALL) , may  be 
specified  as  a dataset  name  (either  in  the  PARM 
field  or  in  file  SYSIN)  to  Indicate  that  all  data- 
sets belonging  to  the  specified  userid  are  to  have 
their  expiry  dates  Increased. 

(c)  File  ARCHCAT  - the  archive  catalogue 
(SYSl.ARCHI VE.CATLG) 

Output  format  - 
(a)  File  SYSPRIN 

Messages  Indicating  the  result  of  the  request  are 
printed  for  each  dataset. 
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(4)  F0RCE2 

calls  - PROTAPD^LASTACC^ELAPSED, JULIAN, ARCHSET, UPDATE, 
LNKMOVE,ENanEQ,SECSPCE,DSNCAT,DELDSN,DSNVTOC 
load  module  - ARCHF2  (must  be  authorized  and  must  be  a 
privileged  program,  with  the  bypass-password-protection 
status ) . 

F0RCE2  Is  the  program  that  performs  user  demand 
archival  and  backup.  It  firstly  determines  the  dataset's 
position  (via  DSNCAT)  and  characteristics  (via  DSNVTOC). 
Only  sequential  or  partitioned  datasets  can  be  archived. 
If  the  dataset  Is  protected  then  the  control  password 
must  be  supplied.  If  not  the  request  is  terminated. 
However,  If  all  conditions  are  satisfied  F0RCE2  generates 
lEHMOVE  control  statements  to  copy  the  dataset  to  tape. 

The  dataset  Is  then  copied  to  the  next  sequence  number 
on  the  archive  tape  (entry  LNKMOVE).  Finally,  on 
successful  completion  of  the  copy,  a record  Is  added  to, 
or  replaced  in,  the  archive  catalogue  and  the  dataset 
deleted  from  disk  (archive  only). 

The  archive  tape  serial  number  and  the  number  of 
datasets  on  the  tape  are  obtained  from  a file  created 
originally  by  Stage  2 of  the  automatic  migration  system. 
This  Information  is  updated  by  F0RCE2  upon  successful 
transfer  of  the  new  dataset  to  the  tape. 

Input  formats  - 

(a)  FARM  field 

dsname,  /ARCHA  months, /password! 

\back/  \ / 

ARCH  indicates  an  archive  request,  BACK  a backup 
request . 

months  is  the  initial  retention  period  in  months, 
indicates  that  no  password  was  supplied. 

(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl.ARCHI VE.CATLG) 

(c)  File  DIREC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

(d)  File  ARCHTAP  - member  ARCHTAP  of 

SYSl. ARCHIVE. PARMLI B.  See  the  description  In  file 
ARCHTAP  of  program  ARCHIVE  (11.2.2(1)). 

Output  formats  - 

(a)  File  lEHMOV 

An  lEHMOVE  control  statement  is  generated  here. 

(b)  File  SYSPRIN 

A message  indicating  the  result  of  the  request  is 
written  here. 
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(5)  POSTDS 

type  - PL/ I 

called  from  - ARCHGOG  (via  a LINK  macro) 
calls  - ENQDEQ/JULIAN 
load  module  - ARCHPOST 

This  program  performs  the  user  requested  migration 
function.  It  turns  on  the  migrate  bit  (bit  6 of  the  flag 
byte)  and  sets  the  Initial  retention  period  (bits  2 to  6) 
In  the  datasets'  entries  In  the  last  access  dataset 
(SYSl. DATASET. LASTACC)/  Indicating  to  the  automatic 
migration  software  that  these  datasets  are  to  be 
migrated.  An  entry  Is  created  for  any  dataset  that  does 
not  already  have  one.  The  dataset  names  may  be  Input 
via  file  SYS  IN  and/or  through  the  PARM  field  (one  name 
only).  The  Initial  retention  period  may  only  be 
specified  In  the  PARM  field  and  applies  to  all  datasets. 

Input  formats  - 

(a)  PARM  field 
/dsname\,  months 
\ NONE  / 

NONE  Indicates  that  no  dataset  name  is  specified 
In  the  PARM  field. 

(b)  File  SYSIN 

Additional  dataset  names,  one  per  80-byte  record 
(starting  in  position  1),  may  be  supplied  here. 

(c)  File  LASTACC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

Output  format  - 
(a)  File  SYSPRINT  - 

A message  verifying  the  request  is  printed  for  each 
dataset . 
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(6)  ARCHGDG 

typo  - nssemhier 

calls  - POSTDS  (via  a LINK  macro  to  modulo  ARCHPOST) 
load  module  - ARCHGDG 

ARCHGDG  performs  user  requested  migration  of  generations 
of  a generation  data  group  (GDG).  The  Input  Indicates 
the  Initial  retention  period  and  the  relative  generation 
number  and  the  program  determines  the  fully  qualified 
name  of  the  dataset  (using  the  LOCATE  macro).  Once  this 
has  been  determined  the  standard  migration  program 
(POSTDS)  Is  Invoked,  using  Its  load  module  name 
(ARCHPOST). 

Input  format  - 
(a)  PARM  field 

dsname,  months 

The  dataset  name  Is  In  the  form  dsn(n),  where  n Is 
the  relative  generation  number. 
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(7)  LSTUSER 
type  - PL/ I 

calls  - ENQDEQ^JULIAN 
load  module  - ARCHLSTU 

Thts  program  Is  Invoked  to  list  the  names  and  expiry 
dates  of  those  datasets  belonging  to  a user  that  are 
currently  In  the  archives. 

Input  formats  - 

(a)  FARM  field 

The  userid  Is  Input  via  the  FARM  field. 

(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

Output  format  - 
(a)  File  SYSPRINT 

The  user  report  Is  output  to  this  file. 

(8)  CHCKDSN 
type  - PL/ 1 
calls  - ENQDEQ 

load  module  - ARCHCHCK 

This  module  Is  used  only  In  the  RETRIEVE  and  RELOAD  TSO 
command  procedure,  (see  Appendix  I I 1.3).  It  runs  In  the 
foreground  and  checks  that  the  dataset  supplied  Is 
actually  In  the  archives.  If  so  the  batch  job  to 
perform  the  data  transfer  Is  submitted.  Otherwise  the 
program  prints  a message  and  sets  a return  code  of  20, 
which  causes  the  command  procedure  to  terminate.  Without 
this  Initial  check  on  the  dataset  name  the  user  would 
have  to  wait  considerably  longer  for  the  batch  job  to 
detect  the  error. 

Input  formats  - 

(a)  FARM  field 

The  dataset  name  Is  Input  In  this  field. 

(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

Output  format  - 
(a)  File  SYSPRINT 


The  error  message  Is  written  to  this  file  If 

nproccA  rsi  . 


55 


WRE-TR-1870{A) 


M.2.5  Archive  tape  transcription  modules 


(1)  TAPEMAP 

type  - PL/ I 

This  program  sorts  the  archive  catalogue  by  archive  tape 
serial  number  and  dataset  sequence  number  and  produces 
a report  Indicating  the  live  datasets  on  each  tape  and 
the  disk  space  they  represent.  From  this  report  tapes 
can  be  selected  for  transcription. 


Input  format  - 

(a)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 


Output  format  - 
(a)  File  SYSPRINT 

The  archive  contents  report  is  written  to  this  file. 


(2)  TAPMRGE 

type  - PL/ I 

This  program  generates  a job  stream  to  perform  the  tape 
transcription.  The  Input  Is  a list  of  the  archive  tape 
serial  numbers  to  be  transcribed  and  a list  of  the  output 
tapes  for  the  transcription  process.  Additional 
parameter  Input  Indicates  the  maximum  number  of  datasets 
that  will  be  written  to  each  output  tape  and/or  the 
maximum  disk  space  that  datasets  on  an  output  tape 
may  represent. 

The  program  extracts  the  records  for  each  dataset  on 
the  input  tapes  from  the  archive  catalogue  and  sorts  them 
on  dataset  sequence  number  within  tape  serial  number. 

The  sorted  records  are  then  reread  and  an  I EHMOVE  job 
stream  (Including  JCL  statements)  is  generated  to  copy 
each  Input  dataset  In  turn  to  an  output  tape.  When  an 
output  tape  becomes  "full"/  as  defined  by  the  parameter 
Input/  the  next  one  Is  used.  The  program  also  writes 
updated  archive  records  (reflecting  the  new  archive 
tape  and  serial  number)  to  a change  accumulation  dataset. 
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Input  formats  - 

(a)  FARM  field 
/max.sets\,/max.  tracks\ 

\ 0 / \ 0 / 

These  fields  Indicate  the  maximum  datasets  and/or 
tracks  each  output  tape  may  contain.  A value  of 
zero  Indicates  that  only  the  other  criterion  Is 
to  be  used. 

(b)  File  ARCHCAT  - the  archive  catalogue 
(SYSl.ARCHIVE.CATLG) 

(c)  File  SYSIN 

Each  80-byte  record  specifies  a single  input  or 
output  tape  serial  number  or  a range  of  them.  A 
maximum  of  50  serial  numbers  may  be  specified  for 
Input  and  for  output. 


Offset  Size 


Descript  ion 


0 

1 

2 

8 


9 


15 


1 'I*  (input)  or  'O'  (output) 

1 unused 

6 tape  serial  number 

1 blank  (indicating  a single 

serial  number  only)  or  '-' 
(indicating  a range) 

6 tape  serial  number  (optional). 

If  '-'  was  specified  in  the 
previous  field  then  this  serial 
number  represents  the  upper  limit 
of  the  range  begun  by  the  first 
serial  number  (e.g.  1 910011- 
910019).  Otherwise  this  field 
contains  blanks. 

65  blank. 


Output  formats  - 

(a)  File  lEHMOVE 

The  lEHMOVE  job  stream  is  generated  here. 

(b)  File  CATCHNG 

The  80-byte  updated  archive  catalogue  records  are 
written  to  this  dataset. 

(c)  File  SYSPRINT 

The  input  parameters  and  exception  messages  are 
wr i tten  to  this  file. 
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(3)  ARCHUP 

type  - PL/ I 
calls  - USERSMF 

load  module  - ARCHUP  (must  be  authorized)  \ 

After  the  I EHMOVE  job  stream  generated  by  TAPMRGE  has 
been  run^  ARCHUP  updates  the  archive  catalogue  with  the 
records  passed  to  it  in  the  change  dataset.  All  user 
changes  to  the  records  affected  since  the  running  of 
TAPMRGE  are  honoured  In  preference  to  those  indicated 
in  the  change  accumulation  dataset. 

Input  formats  - 

(a)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

(b)  File  CATCHNG  - the  change  records  generated  by 
TAPMRGE. 

Output  formats  - 
(a)  File  SYSPRINT 

A message  indicating  the  success  or  failure  of 
each  update  is  written  to  this  file. 
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11.2.6  Archive  cetaloRue  maintenance  modules 

(1)  DUMPSMF 
type  - PL/ I 

calls  - USERSMF^ENQDEQ 

load  module  - ARCHDUMP  (must  be  authorized) 

This  program  obtains  exclusive  use  of  the  archive 
catalogue  and  reads  It  sequentially,,  generating  an  SMF 
record  type  134  (see  Section  11.1(13))  for  each 
catalogue  record.  The  purpose  Is  to  provide  an  extra 
level  of  backup  for  the  catalogue.  Should  both 
SYSl. ARCHIVE. CATLG  and  SYSl.ARCHI VE. COPYCAT  be  destroyed 
an  old  copy  can  be  reconstructed  from  the  SMF 
Information.  In  addition  these  records  provide  a useful 
statistical  history  of  the  size  of  the  archives. 

Input  format  - 

(a)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

(2)  SMFUSER 
type  - PL/ I 

This  program  extracts  the  migration  scheme  SMF 
records  from  an  SMF  dataset  and  writes  them  to  an 
output  dataset. 

During  this  process  change  activity  Is  accumulated  by 
type  and  statistics  are  printed  for  each  day  represented 
by  the  input  SMF  dataset.  Optional  parameter  Input  may 
be  specified  to  list  each  activity  reflected  by  the 
SMF  records  (LIST)  or  not  (NOLIST  - the  default). 

In  addition  a day  range  may  be  specified  to 

instruct  the  program  to  ignore  SMF  records  outside  this 

range . 


Input  formats 
(a)  PARM  field 


/LIST  \ rdatel-date2l 

Xnolist/*-  •* 

The  dates  are  in  Julian  form. 


(b)  File  SMF IN 

All  types  of  SMF  records  generated  by  the 
Operating  System. 


Output  formats  - 
(a)  File  SMFOUT 

The  migration  scheme  SMF  records 
(types  128  to  134). 


(b)  File  SYSPRINT 
The  statistics 


are  printed  on  this  file. 
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(3)  RESTORE 

type  - PL/ I 

RESTORE  uses  the  migration  scheme  SMF  records  passed  to 
it  by  SMFUSER  to  update  a backup  copy  of  the  archive 
catalogue  or  create  an  entirely  new  one  when  the  original 
has  been  lost.  The  type  of  recovery  (CREATE  or  UPDATE) 
and  a time  stamp  (date  and  time)  must  be  specified  via 
parameter  input.  For  an  UPDATE  operation  the  time  stamp 
indicates  the  exact  time  at  which  the  archive  catalogue 
copy  was  taken.  The  program  then  applies  all  updates 
represented  by  SMF  record  types  128,  130,  131,  132  and 
133  generated  since  this  time  to  produce  the  new 
catalogue . 

For  a CREATE  operation  the  time  stamp  indicates  a time 
between  the  last  occasion  the  entire  catalogue  was  copied 
to  SMF  (by  program  DUMPSMF)  and  the  previous  one. 

Starting  from  this  point  the  program  disregards  all 
records  until  It  finds  the  first  type  134.  It  then  adds 
all  records  contained  In  the  string  of  type  134's  to  the 
catalogue,  producing  a basis  for  update.  Next  the 
changes  represented  by  subsequent  type  128,  130,  131, 

132  and  133  SMF  records  are  included. 

Input  formats  - 

(a)  PARM  field 

date,  time,  /CREATE\ 

VUPDATE/ 

The  date  is  in  Julian  form  and  the  time  of  day 
is  In  1/100  th's  of  a second. 

(b)  File  ARCHCAT  - the  archive  catalogue  copy  (for  an 
UPDATE  operation)  or  an  empty  but  initialized 
catalogue  (for  a CREATE  operation) 

(SYSl. ARCHIVE. CATLG) 

(c)  File  SMF 

These  are  the  SMF  records  (types  128  to 
134)  extracted  by  the  program  SMFUSER 

Output  format  - 
(a)  SYSPRINT 

If  any  update  fails  a message  is  written  to  this 
file. 
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(4)  UPCAT 

type  - PL/ I 

calls  - USERSMF^ENQDEQ 

This  program  adds  records  to  the  archive  catalogue. 

It  is  Intended  primarily  to  restore  entries  for  datasets 
mistakenly  deleted  or  let  expire  by  users. 


Input  formats  - 

(a)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

(b)  File  SYSIN 

The  entries  to  be  added  to  the  catalogue  are 
specified  here#  as  80-byte  records. 


Offset  Size 


Descript  ion 


0 


1 


1  a number  from  1 to  6 indicating 

one  of  the  6 possible  values  of 
the  flag-byte. 

1 represents  '00000000'  B - 
partitioned  dataset  unprotected 

2 represents  '10000000'  B - 
sequential  dataset  unprotected 

3 represents  '01100000'  B - 
partitioned,  with  VTOC  protect- 
bits  'll'  B (password  for  write 
but  not  read) 

4 represents  '11100000'  B - 
sequential,  protect  bits  'll’  B 

5 represents  '01000000'  B - 
partitioned,  protect  bits  '10'  B 
(password  for  read  or  write) 

6 represents  '11000000'  B - 
sequential,  protect  bits  '10'  B 

79  the  remaining  79  characters  of  the 
catalogue  record  (see  Appendix 
1.1). 


Output  format  - 
(a)  SYSPRINT 

If  any  addition  fails  a message  is  written  to 
this  file. 


FT 
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11.2.7  Miscellaneous  modules 
(1) 


USERUP 
type  - PL/ I 

This  program  performs  the  maintenance  of  the  user  details 
dataset  (SYSl. USERS) . Records  may  be  added,  deleted  or 
amended . 


Input  formats 
(a)  PARM  field 

[load] 

If  LOAD  is  specified  then  this 
load  of  the  file  SYSl. USERS. 


run  Is  the  initial 


(b) 

(c) 


File  USERS  - the  users'  details  file  (SYSl. USERS) 


File  SYSIN 

There  is  one  80  byte  record  for  each  transaction. 
For  an  update  all  details  must  be  specified,  as  for 
an  addition.  For  a deletion  only  the  userid  is 


requl red. 

Offset 

S i ze 

Desc  r I pt I on 

t 

0 

1 

transaction  type  - 

1 (insert), D (delete), A (alter) 

1 

1 

unused 

2 

8 

user i d 

1 

10 

1 

unused 

11 

20 

user's  name  (5  bytes  for 
initials  then  15  for  surname) 

51 

1 

unused 

32 

10 

user's  group 

i 

42 

1 

unused 

43 

12 

user's  building 

55 

1 

unused 

1 

i 

56 

4 

user's  phone 

1 

60 

1 

unused 

61 

16 

variable  information  (currently 

i 

3 

only  the  first  3 bytes  are 
used  for  the  user's  cost 
centre ) 

76 

4 

unused . 

¥ * 

1 

1 

Output 

fo 

rmat  - 

• 

(a)  File 

SYSPRINT 

1 

Only 

error  and 

exception  messages  are  written 

to 

this  f 1 le. 
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(2)  LISTDSN 

type  - PL/ I 

calls  - ENQDEQ^GETVTOC, ELAPSED, JULIAN 
This  program  creates  a file  with  one  record  for  each 
dataset  currently  on  disk.  Each  record  contains  the 
dataset  name,  the  volume  It  resides  on,  the  size  In 
tracks  and  the  days  since  It  was  last  accessed.  This 
Information  Is  used  as  Input  to  the  migration  scheme 
modelling  program. 

Input  formats  - 

(a)  File  VTOC 

A file  of  154-byte  records  containing  dataset 
names  and  VTOC  Information  as  output  In  fMe 
SORTOUT  by  program  VTOCDSN. 

(b)  File  DIREC  - the  last  access  dataset 
( SYSl . DATASET . LASTACC ) 

Output  format  - 
(a)  File  DSNLIST 

One  80-byte  record  for  each  user  dataset  on 
disk.  In  dataset  name  sequence. 


Offset 

SI  ze 

Description 

0 

8 

userid 

8 

44 

dataset  name 

52 

6 

disk  volume  code 

000001  for  SAOOOl 

000002  for  SA0006 

000003  for  SA0002 

000004  for  SA0005 

000005  for  SA0003 

000006  for  SA0004 

58 

5 

size  in  tracks  (9(5)  format 

63 

5 

days  since  last  accessed 

(9(5)  format) 

68 

12 

unused 

In  addition. 

the  first 

record  In  the  dataset  Indica 

the  date  the 

program  was  run. 

Offset 

Size 

Descr I pt 1 on 

0 

5 

unused  (blank) 

5 

5 

date  ( Jul I an  form) 

10 

4 

unused  (blank) 

14 

6 

date  (numeric  - yymmdd) 

20 

60 

unused 
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(3)  LSTADSN 
type  - PL/ I 

calls  - ELAPSED, JULI AN, ENQDEQ 

LSTADSN  creates  a file  with  a record  for  each  dataset 
currently  In  the  archives.  Each  record  contains  the 
dataset  name,  the  disk  volume  It  originally  resided  on. 
Its  size  In  tracks,  the  number  of  days  since  It  was  last 
accessed  and  the  expiry  date.  This  Information  Is  also 
used  by  the  migration  scheme  modelling  program. 

Input  format  - 

(a)  File  ARCHCAT  - the  archive  catalogue 
(SYSl. ARCHIVE. CATLG) 

Output  format  - 
(a)  File  OUTFILE 

One  80-byte  record  for  each  dataset  In  the  archives, 
sorted  by  dataset  name.  In  addition  the  first 
record  In  the  dataset  Indicates  the  date  the  program 
was  run.  This  date  record  has  the  same  format  as 
that  In  file  SORTOUT  of  program  LISTDSN.  The  data- 
set records  also  have  the  same  format,  up  to  offset 
68,  where  one  extra  field  Is  located. 

Offset  Size  Description 

68  5 expiry  date  (9(5)  format) 

73  7 unused 

(4)  ABEND 

type  - PL/ I 

load  module  - ARCHABND 

ABEND  simply  accepts  a message  of  up  to  100  characters 
from  the  PARM  field  and  writes  it  to  an  output  file. 

The  primary  use  Is  in  conveying  a message  to  users  in 
the  event  of  a data  transfer  request  abending. 

Input  format  - 
(a)  PARM  field 

The  100  character  message  is  Input  here. 

Output  format  - 
(a)  File  SYSPRIN 

The  program  writes  the  message  to  this  file. 
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APPENDIX  III 

MIGRATION  SYSTEM  PROCEDURES 
1 1 1.1  Weekly  migration  catalogued  procedures 
(1)  WARNLIST 

This  procedure  performs  Stage  1 of  the  weekly  migration 
process . 

The  parameters  that  may  be  specified  are; 

(a)  CLEAN  - the  percentage  free  disk  space  required 

(optional/  default*25) 

(b)  WEEKS  - the  maximum  number  of  weeks  datasets  may 

remain  unaccessed  without  being  selected 
(optional/  defau1t»8) 

(c)  DAYS  - the  number  of  days  before  the  datasets  will  be 

transferred  to  the  archives  (I.e.  before 
ARCHLIST  Is  run)  (optional/  default=2) 

The  steps  In  the  procedure  are: 

(a)  COPYACC  - An  I DCAMS  step  to  delete  and  recreate  the 

backup  copy  of  the  last  access  dataset 
( SYSl. DATASET. LASTACC. BACKUP) . The  records 
from  SYSl. DATASET. LAST ACC  are  then  copied 
Into  the  backup  dataset  and  the  former 
deleted  and  reallocated. 

(b)  DSN  - This  step  extracts  dataset  names  and  VTOC 

Information  and  sorts  the  records  on  dataset 
name  (main  program  VTOCDSN/  load  module 
ARCHVDSN) . 

(c)  SMF  - This  step  extracts  the  dataset  access  records 

from  the  SMF  data  and  sorts  them  by  date 
within  dataset  name  (main  program  SMF1415/ 
load  module  ARCHSMFD). 

(d)  UPDTE  - A step  to  recreate  SYSl . DATASET . LASTACC  from 

the  backup  copy  and  dataset  and  access 
Information  generated  In  steps  DSN  and  SMF 
(main  program  DIRECUP/  load  module  ARCHUPDD). 

(e)  WARN  - This  Is  the  major  step  of  the  procedure.  It 

performs  the  dataset  selection  (main  program 
WARNING/  load  module  ARCHWARN). 


i 


I 
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(f)  PRINT  - 


This  step  generates  the  warning  notices  to  be 
sent  to  the  users  responsible  for  the  selected 
datasets  as  well  as  a list  for  the  dataset 
manager  (main  program  LSTWARN/  load  module 
ARCHLSTW) . 


(g)  MTH  - This  step  determines  whether  the  current  day 

Is  In  the  first  week  of  a new  month  and  sets 
the  condition  code  depending  on  the  result 
(main  program  NEWMTH  - see  below). 

(h)  SPLIT  - If  step  MTH  Indicates  that  the  current  day  is 

In  the  first  week  of  a month  this  step 
separates  the  data  on  the  current  SMF  tape 
and  produces  two  new  ones.  One  contains  all 
the  data  for  the  previous  month  and  the 
other  the  data  for  the  current  month  to  date 
(main  program  SMFMTHS  - see  below). 

(I)  CMPRESS  - This  step  generates  a job  stream  to 

compress  all  partitioned  datasets  selected 
for  migration.  The  result  of  the  compress 
highlights  any  problem  datasets  (main 
program  WARNCP  - see  below). 


The  last  three  steps  perform  functions  that  are 
not  strictly  part  of  the  migration  system.  The 
source  modules  of  the  three  programs  are  contained  In 
SYSl.OPS.PLI  and  the  load  modules  In  OPS. LOAD.  A 
brief  description  of  the  programs  Is  given  below. 

NEWMTH 

This  program  checks  the  current  date  and  If  the 
day  number  Is  less  than  or  equal  to  7 sets  the 
condition  code  to  1.  Otherwise  It  Is  set  to  0. 


SMFMTHS 

calls  - JULIAN 

This  program  reads  the  current  generation  of  the 
SMF  history  dataset  and  writes  those  records 
generated  during  the  previous  month  to  one  output 
dataset  and  forms  the  next  generation  of  the 
history  dataset  from  this  month's  records.  The 
program  is  executed  only  If  the  previous  job  step 
returned  a condition  code  of  1. 


WARNCP 

This  Is  basically  the  same  program  as  that  used  in 
the  compress  procedure  (reference  12)#  except  that 
only  those  partitioned  datasets  selected  for 
migration  are  compressed. 
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Should  the  WARNLIST  procedure  need  to  be  restarted 
from  a particular  step  or  even  completely  rerun  care 
should  be  taken  that  the  step  SMFMTHS  Is  executed 
once  and  only  once  each  month.  If  restart  Is 
necessary  the  parameter  DAYS  may  also  need  to  be 
adjusted. 


(2)  ARCHLIST 

This  procedure  performs  Stage  2 of  the  weekly  migration 
process  and  Is  normally  run  2 days  after  WARNLIST. 

The  parameters  available  for  controlling  the  procedure 
are : 

(a)  TAPE  - the  serial  number  of  the  new  archive  tape 

(mandatory) . 

(b)  MTHS  - the  retention  period  to  be  applied  to  the 

newly  migrated  datasets  (optional,  default«3). 

(c)  SYS  - specifies  whether  the  complete  archive 

catalogue  is  to  be  printed  for  the  dataset 
manager's  Information  (Y)  or  not  (N) 

(optional,  default»Y). 

(d)  USER  - specifies  whether  an  archives  status  report 

should  be  produced  for  all  users  (FULL)  or 
only  for  those  who  have  had  a dataset 
migrated  this  run,  have  had  a dataset  expire 
during  the  last  week  or  have  data  sets  with 
less  than  one  month  until  expiry  (PART) 
(optional,  defaul t=PART) . 

The  steps  In  the  procedure  are: 

(a)  HEADER  - This  step  writes  the  dummy,  expiry  date  and 

password  protected  dataset  SYSl . XXnnnnnn  to 
the  new  archive  tape  (where  nnnnnn  is  the 
serial  number). 

(b)  COPYACC  - An  I DCAMS  step  to  delete  and  recreate  the 

backup  copy  of  the  last  access  dataset 
(SYSl. DATASET. LASTACC. BACKUP) . 

(c)  COPYCAT  - An  I DCAMS  step  to  delete  and  recreate  the 

backup  copy  of  the  archive  catalogue 
( SYSl . ARCH  I VE . COPYCAT ) . 

(d)  COPYPARM  - An  I EBCOPY  step  to  create  a backup  copy  of 

the  migration  system  parameter  dataset 
( SYSl . ARCH  I VE . PARML I B . BACKUP ) . 

(e)  DSN  <*  This  program  extracts  dataset  names  and  VTOC 

information  and  sorts  the  records  on  dataset 
name  (main  program  VTOCOSN,  load  module 
ARCHVDSN). 


(f)  ARCH 


This  Is  the  main  step  of  the  procedure.  It 
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generates  utility  control  statements  to  move 
datasets  from  disk  to  tape  (main  program 
ARCHIVE,  load  module  ARCHIV). 

(g)  PROGM  - This  step  executes  an  authorized  and 

privileged  version  of  1 EHPR06M  to  delete  and 
uncatalogue  empty  datasets  that  would  other- 
wise have  been  migrated. 

(h)  MOVE  - An  1 EHMOVE  (authorized  version  with  bypass- 

password-protection  privilege)  step  to  move 
the  datasets  from  disk  to  magnetic  tape. 

(I)  PRINT  - In  this  step  the  users'  archive  status 
reports  and  dataset  manager  reports  are 
produced  (main  program  LSTARCH,  load  module 
ARCHLSTA) . 

Note  that  the  dummy  dataset  created  In  step  HEADER  Is 
not  catalogued  until  the  final  step  of  the  procedure. 

This  Is  Important  should  ARCHLIST  need  to  be 
restarted,  which  may  be  done  from  most  steps. 
Detailed  considerations  for  restarting  WARNLIST  and 
ARCHLIST  are  contained  In  a separate  set  of  notes 
distributed  to  the  relevant  operations  duty 
programmers . 

1 1 1.2  User  facility  catalogued  procedures 

The  syntax  for  using  these  procedures  will  not  be 
discussed  here.  Reference  11  should  be  consulted  for  these 
details.  Section  5 describes  the  purpose  of  the  procedures. 

(1)  RETRIEVE  and  RELOAD 

The  parameters  available  are: 

(a)  DSN  - the  dataset  to  be  retrieved  or  reloaded 

(mandatory) . 

(b)  PASSWD  - the  dataset's  control  password  (optional, 

required  only  If  the  dataset  Is  protected). 
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The  steps  in  these  procedures  are: 


(a)  ALLOC 

(b)  ARCHIV 

(c)  PR06M 


This  step  allocates  the  temporary 
datasets  used  (program  IEFBR14). 

This  Is  the  main  step.  It  performs  what- 
ever checks  are  necessary  and  Invokes 
lEHMOVE  to  copy  the  dataset  to  disk  (main 
program  RETRVE,  load  module  ARCHRET). 

Note  that  there  must  be  a DO  card  In  this 
step  for  each  disk  Involved  In  the 
migration  scheme. 

An  lEHPROGM  step  to  catalogue  the 
retrieved  dataset. 


(d)  UNALLOC  - Another  IEFBR14  step  to  delete  the 
temporary  datasets. 


(e)  ABND1^ABND2  - Steps  to  issue  a message  to  the  user 

In  case  of  an  abend  (main  program 
ABEND,  load  module  ARCHABND). 


(2)  ASCRATCH 
Parameters  available: 

(a)  DSN  - a dataset  to  be  deleted  from  the  archives 

(opt  I onal ) . 

(b)  PASSWD  - the  dataset's  control  password  (optional). 

The  only  step  In  procedure  ASCRATCH  is  - 

(a)  SCR  - This  step  performs  password  checking  and  If 
successful  deletes  the  specified  datasets 
(main  program  SCRATCH,  load  module  ARCHSCR). 
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(3)  EXPIRY 

The  parameters  available  are: 


(a) 

DSN  - 

a dataset  whose  expiry  date 
extended  (optional). 

Is 

to  be 

(b) 

ADD  - 

the  number  of  months  to  add 
(optional/  default>6). 

to 

the  date 

The  only  step  In  procedure  EXPIRY  Is  - 

(a)  EXP  - this  step  updates  the  expiry  date  field  in 
the  archive  catalogue  records  of  the 
specified  datasets  (main  program  EXPIRY/ 
load  module  ARCHEXP). 


(4)  ARCHIVE  and  BACKUP 
Parameters  available: 

(a)  DSN  - the  dataset  to  be  copied  to  archives 

(mandatory) . 

(b)  MTHS  - the  Initial  retention  period 

(optional/  default"6). 

(c)  PASSWD  - the  dataset's  control  password  (optional). 
The  steps  In  the  procedures  are  - 

(a)  ALLOC  - An  IEFBR14  step  to  allocate  the  temporary 

datasets  required. 

(b)  ARCHIV  - This  Is  the  major  step.  It  uses  catalogue 

and  VTOC  Information  to  generate  utility 
control  statements  to  transfer  the  dataset 
to  magnetic  tape  and  invokes  I EHMOVE  to 
perform  the  operation  (main  program  F0RCE2/ 
load  module  ARCHF2).  Note  that  there  must 
be  a DD  card  In  this  step  for  each  disk 
Involved  In  the  migration  scheme. 

(c)  UNALLOC  - An  IEFBR14  step  to  delete  the  temporary 

datasets . 

(d)  ABND1/ABND2  - Steps  to  Issue  a message  to  the  user  In 

case  of  an  abend  (main  program  ABEND/ 
load  module  ARCHABND). 
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(5)  MIGRATE 

The  parameters  available  are: 

(a)  DSN  - a dataset  name  (optional). 

(b)  MTHS  the  Initial  retention  period  (optional/ 

default  ■ 6). 

The  only  step  In  the  procedure  Is: 

(a)  MIG  - For  each  specified  dataset  this  step  sets  the 
migrate  bit  and  retention  period  bits  in  the 
entry  In  SYSl. DATASET. LASTACC/  or  creates 
a new  entry  If  one  does  not  already  exist 
(main  program  POSTDS/load  module  ARCHPOST). 


(6)  GM I GRATE 

The  parameters  available  are: 

(a)  DSN  - the  name  of  the  relative  generation  to  be 

migrated  (mandatory). 

(b)  MTHS-  the  initial  retention  period  (optional/ 

defaul t»6) . 


(a)  MIG  - Determines  the  absolute  generation  number  (from 
the  catalogue)  and  hence  the  true  dataset  name. 
It  then  Invokes  ARCHPOST  to  perform  the 
migration  (main  program  ARCHGDG/  load  module 
ARCHGDG) . 


(7)  LI  STARCH 

The  parameter  used  by  this  procedure  is: 

(a)  USER  - the  userid  whose  archive  status  report  is 
requ I red. 

The  single  step  Is  - 

(a)  LIST  - In  this  step  the  user's  entries  In  the  archive 
catalogue  are  located  and  printed  (main 
program  LSTUSER/  load  module  ARCHLSTU). 
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111,3  User  facility  convnand  procedures 

Each  of  the  catalogued  procedures  described  In  1 1 1.3  has 
an  equivalent  TSO  command  procedure  of  the  same  name.  These 
are  stored  In  the  dataset  SYSl.CLIST. 

The  procedures  MIGRATE/  GMIGRATE  and  LI  STARCH  execute  as  a 
foreground  job.  The  remainder  edit  the  appropriate  member  of 
SYSl. ARCHIVE. CNTL  and  submit  the  JCL  to  perform  the  request 
as  a batch  job  in  the  F or  H Initiator.  This  is  necessary 
because  these  procedures  Involve  either  a tape  mount  and/or 
execute  an  authorized  load  module/  both  of  which  are 
prohibited  from  the  foreground  under  the  W.R.E. 
Implementation  of  TSO. 

In  addition  both  RELOAD  and  RETRIEVE  execute  another 
program  (CHCKDSN  - see  Appendix  11.2.4(a))  in  the  foreground 
before  submitting  the  batch  job.  This  program  checks  that 
the  dataset  Is  In  the  archives.  If  so  the  job  is  submitted. 

If  not  the  user  receives  an  error  message  immediately  and  the 
job  Is  not  submitted.  Without  this  check  users  would  have  to 
wait  for  a batch  job  to  execute  before  such  errors  were 
detected. 
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APPENDIX  IV 


SMF  RECORD  TYPE  140  FORMAT 


Offset 

Size 

Description 

0 

1 

system  Indicator  (“X'02'  - VS2) 

1 

1 

record  type  (binary)  ■ 140 

2 

4 

time  of  day  record  was  written  In 
hundredths  of  a second  (binary) 

6 

4 

date  record  was  written  In  Julian 
form  (packed  decimal) 

10 

4 

machine  Identifier  (-CWIOO') 

14 

8 

name  of  Job 

22 

4 

time  of  day  reader  recognized  the 
JOB  card/  In  hundredths  of  a second 
(binary) 

26 

4 

date  reader  recognized  the  JOB  card 
In  Julian  form  (packed  decimal) 

30 

8 

user  Identification  field 

38 

26 

unused  - zeros 

64 

44 

dataset  name 
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TABLE  1.  OPERATING  STATISTICS 


Period  (see  Section  9) 

1 

2 

1 

3 

no.  disks  available 

2 

6 

1 

6 

no.  automatic  migration  runs 

13 

14 

1 

14 

no.  working  weeks  In  period 

13 

16 

1 

14 

no.  datasets  migrated 

827 

1499 

1 

3756 

average  migrations  per  run 

64 

107 

1 

268 

no.  retrievals  or  reloads 

144 

454 

1 

1369 

average  retrievals  per  week 

11 

28 

1 

98 

no.  user  archivals  or  backups 

75 

263 

1 

665 

average  user  archivals/backups 

per  week  | 

6 

16 

1 

48 

no.  deletions  or  expiry  date  lapses  I 

1 

69* 

653 

1 

1233 

average  deletions  per  week 

1 

1 

5* 

41 

1 

88 

no. datasets  In  archives  at  end 

of  perlodi 

1 

741 

1700 

1 

3822 

* 


Note  that 
months  of 


no  datasets  would  have  expired  in  the 
this  period. 
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